Cognition of a Robotic Manipulator Using the Q-Learning Based Situation-Operator Model

Cognition of a Robotic Manipulator Using the Q-Learning Based Situation-Operator Model

Akash Dutt Dubey, Ravi Bhushan Mishra
Copyright: © 2020 |Pages: 13
DOI: 10.4018/978-1-7998-1754-3.ch030
(Individual Chapters)
No Current Special Offers


In this article, we have applied cognition on robot using Q-learning based situation operator model. The situation operator model takes the initial situation of the mobile robot and applies a set of operators in order to move the robot to the destination. The initial situation of the mobile robot is defined by a set of characteristics inferred by the sensor inputs. The Situation-Operator Model (SOM) model comprises of a planning and learning module which uses certain heuristics for learning through the mobile robot and a knowledge base which stored the experiences of the mobile robot. The control and learning of the robot is done using q-learning. A camera sensor and an ultrasonic sensor were used as the sensory inputs for the mobile robot. These sensory inputs are used to define the initial situation, which is then used in the learning module to apply the valid operator. The results obtained by the proposed method were compared to the result obtained by Reinforcement-Based Artificial Neural Network for path planning.
Chapter Preview

1. Introduction

With the evolution of the cognitive applications in robots, the robot manipulators are expected to improve their performance in a wide range of situations. A cognitive robot is expected to be fluent in routine operations and capable to adjusting itself to the changing and random environments. For this purpose, the robot must be able to take decisions according to the environments with the use of sensors which provide it the information of the environment. The ability of the robot manipulator is known as the Cognitive Control which basically works on action-selection process. This cognitive control is similar to the control which is observed in human beings as it takes a set of component behaviors to reach its destination (Ratanaswasd, Dodd, Kawamura, & Noelle, 2005).

The cognition of a robot using computer algorithms and programs are described as Computational psychology tools which are also known as computational cognitive models. Since these models have the ability to generate or predict the behavior of a robot, they can also be applied to obtain the cognitive behavior for the control of unmanned vehicles. There are many types of cognitive models that can be applied to the robots and they have been categorized to two sub categories symbolic and sub symbolic. The difference between the symbolic and sub symbolic systems is the relationship between the levels of semantic interpretation. While in symbolic systems, the level of semantic interpretation and the formal representation for processing are done at the level of symbols, in the sub symbolic systems, the formal representation for processing takes place below the level at which the concepts of interest are described (Hanford, Janrathitikarn, & Long, 2008).

The learning models have become an interesting tool for the robotics, basically because complexity of the robot and its environment is so high that it becomes hard to obtain an accurate analytical model (Nakanishi et al., 2008, Nguyen tuong, Seeger, & Peters, 2009). The cognitive models are characterized by the ability to aggregate the knowledge and design the structure autonomously. Complex technical systems can be enhanced using these cognitive models to adapt the system in open and unknown environment. As stated by Strube (1993), the cognitive models have the capability of autonomous learning and realize the environment based on its knowledge.

For the purpose of providing self-sufficiency to the robots, various cognitive models have been designed which include ACT-R (Anderson, 1996), SOAR (Laird, Newell, & Rosenbloom, 1987) and EPIC (Kieras & Meyer, 1997). Act-R is a cognitive architecture which divided the knowledge in two parts, declarative and procedural. The declarative knowledge was represented in forms of chunks while the procedural knowledge used productions for representation.

An embodied version of ACT-R, ACT-R/E (Trafton et al., 2009) was applied on the robots in order to perform various coordinating tasks with human members. Similarly, Soar architecture also used a production system where the knowledge about the current state is used to decide the behavior of the agent using the production rules. Soar cognitive architecture has been used in controlling mobile robots and also unmanned vehicles (Jones et al., 1999; Smith & Willcox, 2005). Perception and actions have been emphasized in the EPIC (Executive Process-Interactive Control) along with the central cognition for task performance. EPIC architecture entails every detail about processing of sensory information which includes visual, auditory and tactile and also, motor activities. These properties of EPIC enable the architecture for modeling multitasking behavior (Taatgen & Anderson, 2008).

Complete Chapter List

Search this Book: