Classifying Behaviours in Videos with Recurrent Neural Networks

State Of The Art

Human Behaviour Analysis (HBA) involves a wide range of applications: Video Surveillance, Ambient-Assisted Living, etc. All these applications have in common the need of creating an artificial intelligence that understands the body of a person and its natural movement for different activities. Human activities, such as “walking” or “running,” are relatively easy to recognize. On the other hand, more complex activities, such as “peeling an apple,” are more difficult to identify. Complex activities may be decomposed into other simpler activities, which are generally easier to recognize. Therefore, it is necessary to understand the different HBA levels that exist. Moeslund, Hilton, and Krüger (2006) defined a classification of the different action taxonomies that have been adopted later in many other works. It defines three levels of abstractions from smallest to biggest: 1. Action primitive: Basic motion recognition that represents the atomic movement out of which actions are built. 2. Action: Composed of different action primitives. 3. Activity: A higher level of abstraction which requires the semantic notion of the context and the involved objects.

Although this taxonomy is highly used among researchers, some of them use their own taxonomies, for example Ji, Liu, Li, and Brown (2008) include a higher level of abstraction called behaviour. They defined behaviour as” human motion patterns involving high-level description of actions and interactions”.

Motion recognition is the fundament for detecting human activities or behaviours. Motion is decomposed in a series of poses through time. A pose can be described as the state of the body posture that can be represented by an articulated system of rigid segments connected by joints, like the model described in Andriluka, Roth, and Schiele (2009); Sapp, Toshev, and Taskar (2010).