Deep Reinforcement Learning in ROS
Deep reinforcement learning is a combination of deep learning and reinforcement learning. It allows robots to achieve a high degree of autonomy of exploration and navigation (Kim et al., 2016). There are two approaches in deep reinforcement learning mainly (a) Model-based learning (b) Value-based learning. This work focuses on value-based learning, this value decides the next action be taken (Unhelkar et al., 2015).
Figure 1.
Schematic of Deep reinforcement learning model
Figure 1 illustrates the workflow of memory-based deep-reinforcement learning model, where the input is through the sensor database on the depth and range sensing (Gohane & Khekare, 2015). Along with it, autonomous exploration is based on Markov Decision Process (Trautman, 2015). Using DRL it learns to take the optimal decision at time t in a safe state (Kretzschmar et al., 2016). The decision will be taken according to the policy learned by the algorithm. It uses a socially aware collision avoidance algorithm using deep reinforcement learning (Berg et al., 2011).
Where a multi-agent collision avoidance problem is addressed as a sequential decision-making problem in the reinforcement learning framework (Khekare, 2014). Where the agent’s position, velocity, and size is described by a set of states. Also inducing social norms for example the distance to the other agents is taken into consideration. Previous works have reported that the teleoperation of a robot bound to be cooperative and time-efficient (Khekare & Janardhan, 2017).
To solve the problem of collision avoidance, it performs multi-agent value training on a two-agent network, but the drawback of their system is, it can- not be scalable to more than two agents (Khekare et al., 2020). Whereas social norms of the agent’s environment are taken into consideration by the SA-CADRL algorithm (Mehta et al., 2016). Whereas some papers discuss the social force model for pedestrian dynamics. It uses the Deep Q Network (DQN) which is a combination of deep learning and Q-learning thinking is drawing attention as a reinforcement learning algorithm (Berg et al., 2008).
Our Strategy is divided into four parts including Mapping, Localization, Path Planning, and collision avoidance.
Mapping
Mapping is the process of building a map of the floor or the environment where the robot is in tented to move. The map is the representation of the environment created from sensor readings of the robot, here the LIDAR sensor is used based on lased guided navigation (Kim et al., 2018). For mapping turtle, bot navigation package in ROS is used, along with mapping package of gazebo simulation. To map the robot teleop is used so that the robot should the complete floor. Illustrated in Figure 2 and Figure 3.
Localization
In Autonomous Navigation robot has to locate its position concerning the environment, so that is can easily move towards its goal position (Khekare et al., 2019). Localization is the method to find out where the robot is concerning the map. Gazebo AMCL package is used for localization along with turtle bot navigation stack, Illustrated in Figure 4. To view the localization of the robot, RViz visualization package of ROS is used (Chen et al., 2017).