In this chapter, the authors describe a novel approach for inferring intention during cooperative human-robot activities through the representation and ordering of state information. State relationships are represented by a combination of spatial relationships in a Cartesian frame along with cardinal direction information. The combination of all relevant state relationships at a given point in time constitutes a state. A template matching approach is used to match state relations to known intentions. This approach is applied to a manufacturing kitting operation1, where humans and robots are working together to develop kits. Based upon the sequences of a set of predefined high-level state relationships that must be true for future actions to occur, a robot can use the detailed state information presented in this chapter to infer the probability of subsequent actions. This would enable the robot to better help the human with the operation or, at a minimum, better stay out of his or her way.
Top1. Introduction
Humans and robots working safely and seamlessly together in a cooperative environment is one of the future goals of the robotics community (CCC, 2009). When humans and robots can work together in the same space, a whole class of tasks can be automated, ranging from collaborative assembly to parts and material handling to delivery. Keeping humans, who are within a robotic work cell safe, requires the ability of the robot to monitor the work area, infer human intention, and be aware of potential dangers to avoid them. Robots are under development throughout the world that will revolutionize manufacturing by allowing humans and robots to operate in close proximity while performing a variety of tasks (Szabo, 2011).
Proposed standards exist for robot-human safety (such as work performed by International Organization for Standardization (ISO) / Robotics Industries Association (RIA) 10218-2 Safety Requirements – Part 2: Industrial robot systems and integration), but these standards focus on robots adjusting their speed based on the separation distance between the human and the robot (Chabrol, 1987). In essence, as the robot gets closer to a detected human, the robot gradually decreases its speed to ensure that if a collision between the human and robot occurs, minimal damage will be caused. These standards focus on where the human is at a given point in time. It does not focus on where they are anticipated to be in the future.
A key enabler for human-robot safety in cooperative environments involves the field of intention recognition. Intention recognition involves the robot attempting to understand the intention of an agent (the human) by recognizing some or all of his/her actions (Sadri, 2011) to help predict the human’s future actions. Knowing these future actions will allow a robot to plan in such a way as to either help the human perform his/her activities or, at a minimum, not put itself in a position to cause an unsafe situation.
In this chapter, we present an approach to representing state information in an ontology for the purpose of ontology-based intention recognition. An overview of the intention recognition approach can be found in (Schlenoff, 2012a). In this context, we adopt the (Tomasello, 2005) definition of intention as “a plan of action the organism chooses and commits itself to in pursuit of a goal – an intention thus includes both an action plan as well as a goal.” We also distinguish states from state relationships. In this context, we define a state as a set of properties of one or more objects in an area of interest that consist of specific recognizable configurations and or characteristics. A state relationship is a specific relation between two objects (e.g., Object 1 is on top of Object 2). A set of all relevant state relationships in an environment composes a state. This approach to intention recognition is different than many ontology-based intention recognition approaches in the literature (as described in the next section) as they primarily focus on activity (as opposed to state) recognition and then use a form of abduction to provide explanations for observations. We infer detailed state relationships using observations based on Region Connection Calculus 8 (RCC8) (Randell, 1992) and then combine these observations using the Semantic Web Rule Language (SWRL) (W3C_Member_Submission, 2004) to infer the overall state relationships that are true at a given time. Once a sequence of state relationships has been determined, we will use probabilistic procedures to associate those states with likely overall intentions to determine the next possible action (and associated state) that is likely to occur. This chapter focuses on the way that states are represented in the ontology and how intentions can be inferred from them.