Using Haptic Feedback in Human-Swarm Interaction

Using Haptic Feedback in Human-Swarm Interaction

Steven Nunnally (University of Pittsburgh, USA), Phillip Walker (University of Pittsburgh, USA), Michael Lewis (University of Pittsburgh, USA), Nilanjan Chakraborty (Carnegie Mellon University, USA) and Katia Sycara (Carnegie Mellon University, USA)
DOI: 10.4018/978-1-4666-9572-6.ch022
OnDemand PDF Download:
No Current Special Offers


A swarm of robots is a large group of individual agents that autonomously coordinate via local control laws. Their emergent behavior allows simple robots to accomplish complex tasks. Since missions may have complex objectives that change dynamically due to environmental and mission changes, human control and influence over the swarm is needed. The field of Human Swarm Interaction (HSI) is young, with few user studies, and even fewer papers focusing on giving non-visual feedback to the operator. The authors will herein present a background of haptics in robotics and swarms and two studies that explore various conditions under which haptic feedback may be useful in HSI. The overall goal of the studies is to explore the effectiveness of haptic feedback in the presence of other visual stimuli about the swarm system. The findings show that giving feedback about nearby obstacles using a haptic device can improve performance, and that a combination of feedback from obstacle forces via the visual and haptic channels provide the best performance.
Chapter Preview


A robotic swarm consists of a large collection of simple robots with limited sensing, communication, actuation, and computational capabilities. Individual robots act according to simple local rules and exhibit a wide range of behaviors, such as flocking (Reynolds, 1987; Couzin, Krause, James, Ruxton, & Franks, 2002; Spears & Spears, 2012; Bruemmer, 2002) without any centralized controller. However, for performing complex tasks like search and exploration in obstacle-filled environments, it is usually difficult to design local control laws for individual swarms that guarantee good performance of the overall system. To use swarm robotic systems in a complex mission, the presence of human operators are required to guide the behaviors of the swarm towards accomplishing mission goals. A key aspect of using a human to control a swarm is the transfer of information between the human and the swarm. The human has to obtain information about the state of the swarm in order to control it. In the extant literature, experimental studies in human swarm interaction have primarily explored the use of the visual channel of the human to transfer information about the swarm state. However, the use of the haptic channel has not been studied adequately in HSI, except for formation control tasks and with small multi-robot systems (fewer than 10 robots). Therefore, along with background literature, this chapter will present experiments to explore the benefits of using a haptic device to control swarm robots (in addition to the visual channel), using large numbers of robots to demonstrate scalability.

A key aspect of using a haptic device to feed information back about the robot state is to decide on the information (or cue) that should be fed back from the robots to the human. In many swarm robotic algorithms, potential field based methods (LaValle, 2006) are used for avoiding obstacles. Roughly speaking, when a robot is near an obstacle, the robot controller computes a virtual force from the obstacle that is inversely proportional to the distance (or some superlinear function of the distance) between the robot and the obstacle. Thus, the nearer a robot is to an obstacle, the greater the force it “experiences” that makes it move away from the obstacle. Therefore, one cue that can be fed back to the human is the net obstacle forces experienced by the robots.

In formation control tasks, since the robots have to usually maintain a rigid formation as they move, each robot has to track a path in order to maintain a formation. Thus, a natural cue in formation control is the tracking error of the robots in following their paths. The use of net forces from the obstacles along with tracking errors has been explored in the context of formation control (Son et al., 2011). In many applications of swarm systems, especially in obstacle-filled environments, it is not always desirable to move the robots in a formation. In these cases, there is no natural notion of tracking error, so the force cue from the obstacles can be fed back to the humans through the haptic device instead.

Key Terms in this Chapter

Flocking: A swarm behavior whereby all agents align their velocities and headings while maintaining some minimum distance from one another.

Haptic Feedback: Information fed back from a system to a user that is tactile in nature.

Predictive Display: A graphical user interface that displays a prediction about a swarm’s future state, in addition to current information.

Human-Swarm Interaction: A system including a human operator in control of a semi-autonomous swarm. In such a system, the human uses the information returned by the swarm about the environment and swarm itself to give new commands and inputs to the swarm.

Summary Display: A graphical user interface displaying summary information about the swarm state, such as a bounding centroid or average heading.

Swarm: A robust, scalable system composed of numerous robots, which coordinate through local interaction only.

Complete Chapter List

Search this Book: