Multiagent Learning on Traffic Lights Control: Effects of Using Shared Information

Multiagent Learning on Traffic Lights Control: Effects of Using Shared Information

Denise de Oliveira (Universidade Federal do Rio Grande do Sul, Brazil) and Ana L.C. Bazzan (Universidade Federal do Rio Grande do Sul, Brazil)
Copyright: © 2009 |Pages: 15
DOI: 10.4018/978-1-60566-226-8.ch015
OnDemand PDF Download:


In a complex multiagent system, agents may have different partial information about the system’s state and the information held by other agents in the system. In a distributed urban traffic control, where each junction has an independent controller, agents that learn can benefit from exchanging information, but this exchange of information may not always be useful. In this chapter the authors analyze how agents can benefit from sharing information in an urban traffic control scenario and the consequences of this cooperation in the performance of the traffic system.
Chapter Preview


Urban traffic control (UTC) is an important and challenging real-world problem. This problem has several important characteristics, related to its dynamics (changes in the environment are not only consequences of the agents actions, changes are beyond the agents control); to non-determinism (each action may have more than one possible effect); and to partial observability (each agent perceives a limited fraction of the current environment state).

Multiagent learning can be seen as a suitable tool for coping with the issues related to the dynamicity in this scenario. Formalizing the problem of control is an important part of the solution, and the theory of Markov Decision Processes (MDP) has shown to be particularly powerful in that context. Defining the traffic control problem as a single MDP, i.e. in a centralized way, would lead to an unsolvable problem, due the large number of possible states. For instance, consider a scenario where six traffic lights with five possible states each, according to the incoming links (streets) congestion: all links have the same number of stopped vehicles, North link has more waiting vehicles, South link has more waiting vehicles, East link has more waiting vehicles, and West link has more waiting vehicles. In this case, the number of possible states is 15,625 (56) and the number of possible joint actions is 729 (36), considering that each traffic light has three possible actions. The number of Q-values is 11,390,625 (729 x 15,625). In a decentralized solution, traffic controlling agents may have different partial information about the system state and the information held by other agents in the system.

The distributed urban traffic control (DUTC) problem has some important characteristics to be considered: a large number of possible traffic pattern configurations, limited communication, limited observation, limited action frequency and delayed reward information. Delayed reward, since the traffic flow takes some time to respond to the agent’s actions, this time is, at least, the duration of one cycle time.

In this chapter we explore some questions about information shared among learning agents in a traffic scenario. We also discuss the multiagent reinforcement learning used as a solution to the traffic control problem, current limitations and further developments.

Complete Chapter List

Search this Book:
List of Reviewers
Table of Contents
Ana Bazzan, Franziska Klügl
Ana Bazzan, Franziska Klügl
Chapter 1
Takeshi Takama
This study discusses adaptation effects and congestion in a multi-agent system (MAS) to analyse real transport and traffic problems. Both... Sample PDF
Adaptation and Congestion in a Multi-Agent System to Analyse Empirical Traffic Problems: Concepts and a Case Study of the Road User Charging Scheme at the Upper Derwent
Chapter 2
Qi Han, Theo Arentze, Harry Timmermans, Davy Janssens, Geert Wets
Contributing to the recent interest in the dynamics of activity-travel patterns, this chapter discusses a framework of an agent-based modeling... Sample PDF
A Multi-Agent Modeling Approach to Simulate Dynamic Activity-Travel Patterns
Chapter 3
Michael Balmer, Marcel Rieser, Konrad Meister, David Charypar, Nicolas Lefebvre, Kai Nagel
Micro-simulations for transport planning are becoming increasingly important in traffic simulation, traffic analysis, and traffic forecasting. In... Sample PDF
MATSim-T: Architecture and Simulation Times
Chapter 4
Ulf Lotzmann
In this chapter an agent-based traffic simulation approach is presented which sees agents as individual traffic participants moving in an artificial... Sample PDF
TRASS: A Multi-Purpose Agent-Based Simulation Framework for Complex Traffic Simulation Applications
Chapter 5
Paulo A.F. Ferreira, Edgar F. Esteves, Rosaldo J.F. Rossetti, Eugénio C. Oliveira
Trading off between realism and too much abstraction is an important issue to address in microscopic traffic simulation. In this chapter the authors... Sample PDF
Applying Situated Agents to Microscopic Traffic Modelling
Chapter 6
Andreas Schadschneider, Hubert Klüpfel, Tobias Kretz, Christian Rogsch, Armin Seyfried
Multi-Agent Simulation is a general and powerful framework for understanding and predicting the behaviour of social systems. Here the authors... Sample PDF
Fundamentals of Pedestrian and Evacuation Dynamics
Chapter 7
Rex Oleson, D. J. Kaup, Thomas L. Clarke, Linda C. Malone, Ladislau Bölöni
The “Social Potential”, which the authors will refer to as the SP, is the name given to a technique of implementing multi-agent movement in... Sample PDF
"Social Potential" Models for Modeling Traffic and Transportation
Chapter 8
Sabine Timpf
In this chapter, the authors present a methodology for simulating human navigation within the context of public, multi-modal transport. They show... Sample PDF
Towards Simulating Cognitive Agents in Public Transport Systems
Chapter 9
Kurt Dresner, Peter Stone, Mark Van Middlesworth
Fully autonomous vehicles promise enormous gains in safety, efficiency, and economy for transportation. In previous work, the authors of this... Sample PDF
An Unmanaged Intersection Protocol and Improved Intersection Safety for Autonomous Vehicles
Chapter 10
Heiko Schepperle, Klemens Böhm
Current intersection-control systems lack one important feature: They are unaware of the different valuations of reduced waiting time of the... Sample PDF
Valuation-Aware Traffic Control: The Notion and the Issues
Chapter 11
Charles Desjardins, Julien Laumônier, Brahim Chaib-draa
This chapter studies the use of agent technology in the domain of vehicle control. More specifically, it illustrates how agents can address the... Sample PDF
Learning Agents for Collaborative Driving
Chapter 12
Kagan Tumer, Zachary T. Welch, Adrian Agogino
Traffic management problems provide a unique environment to study how multi-agent systems promote desired system level behavior. In particular, they... Sample PDF
Traffic Congestion Management as a Learning Agent Coordination Problem
Chapter 13
Matteo Vasirani, Sascha Ossowski
The problem of advanced intersection control is being discovered as a promising application field for multiagent technology. In this context... Sample PDF
Exploring the Potential of Multiagent Learning for Autonomous Intersection Control
Chapter 14
Tomohisa Yamashita, Koichi Kurumatani
With maturation of ubiquitous computing technology, it has become feasible to design new systems to improve our urban life. In this chapter, the... Sample PDF
New Approach to Smooth Traffic Flow with Route Information Sharing
Chapter 15
Denise de Oliveira, Ana L.C. Bazzan
In a complex multiagent system, agents may have different partial information about the system’s state and the information held by other agents in... Sample PDF
Multiagent Learning on Traffic Lights Control: Effects of Using Shared Information
Chapter 16
Tamás Máhr, F. Jordan Srour, Mathijs de Weerdt, Rob Zuidwijk
While intermodal freight transport has the potential to introduce efficiency to the transport network,this transport method also suffers from... Sample PDF
The Merit of Agents in Freight Transport
Chapter 17
Lawrence Henesey, Jan A. Persson
In analyzing freight transportation systems, such as the intermodal transport of containers, often direct monetary costs associated with... Sample PDF
Analyzing Transactions Costs in Transport Corridors Using Multi Agent-Based Simulation
Chapter 18
Shawn R. Wolfe, Peter A. Jarvis, Francis Y. Enomoto, Maarten Sierhuis, Bart-Jan van Putten
Today’s air traffic management system is not expected to scale to the projected increase in traffic over the next two decades. Enhancing... Sample PDF
A Multi-Agent Simulation of Collaborative Air Traffic Flow Management
About the Contributors