Evaluation of Spatio-Temporal Microsimulation Systems

Evaluation of Spatio-Temporal Microsimulation Systems

Christine Kopp (Fraunhofer Institute for Intelligent Analysis and Information Systems (IAIS), Germany), Bruno Kochan (University of Hasselt, Belgium), Michael May (Fraunhofer Institute for Intelligent Analysis and Information Systems (IAIS), Germany), Luca Pappalardo (ISTI-CNR, Italy & University of Pisa, Italy), Salvatore Rinzivillo (ISTI-CNR, Italy), Daniel Schulz (Fraunhofer Institute for Intelligent Analysis and Information Systems (IAIS), Germany) and Filippo Simini (University of Bristol, UK)
Copyright: © 2014 |Pages: 26
DOI: 10.4018/978-1-4666-4920-0.ch008
OnDemand PDF Download:
No Current Special Offers


The increasing expressiveness of spatio-temporal microsimulation systems makes them attractive for a wide range of real world applications. However, the broad field of applications puts new challenges to the quality of microsimulation systems. They are no longer expected to reflect a few selected mobility characteristics but to be a realistic representation of the real world. In consequence, the validation of spatio-temporal microsimulations has to be deepened and to be especially moved towards a holistic view on movement validation. One advantage hereby is the easier availability of mobility data sets at present, which enables the validation of many different aspects of movement behavior. However, these data sets bring their own challenges as the data may cover only a part of the observation space, differ in its temporal resolution, or not be representative in all aspects. In addition, the definition of appropriate similarity measures, which capture the various mobility characteristics, is challenging. The goal of this chapter is to pave the way for a novel, better, and more detailed evaluation standard for spatio-temporal microsimulation systems. The chapter collects and structure’s various aspects that have to be considered for the validation and comparison of movement data. In addition, it assembles the state-of-the-art of existing validation techniques. It concludes with examples of using big data sources for the extraction and validation of movement characteristics outlining the research challenges that have yet to be conquered.
Chapter Preview


Modeling individual movement behavior is a complex task and requires thorough validation throughout the modeling process. The complexity of spatio-temporal microsimulation systems originates mainly from the vast solution space of individual movements in geographic space and time. The size of the solution space is closely linked to the spatial and temporal resolution of the microsimulation, which is typically predetermined by the application. The outcome of a microsimulation is a complete mobility model for a given region, time period and population. Based on a synthetic population it provides a detailed schedule about who moves when and where using which mode of transport. This comprehensive information makes the validation of spatio-temporal microsimulation systems challenging.

In current practice the evaluation of microsimulations relies on a partial evaluation of mobility characteristics. Most often a comparison with traffic counts is performed as e.g. in (Gao et al., 2010; Horni et al., 2009; Meister et al., 2010). Traffic counts have the advantage that they can be comparably easy obtained. However, traffic counts do not contain origin-destination information and are typically available for vehicular traffic only. Thus, they cover only a small aspect of individual movement. In addition, most traffic counts are available for major roads only and are therefore not representative for the whole street network. In other words, traffic counts are a vital source for the validation of microsimulation systems but their application is limited to a subset of model characteristics.

The example of traffic counts illustrates that a new holistic validation concept for spatio-temporal microsimulation systems is needed. Only a validation which considers a broad set of mobility characteristics can ensure that the outcome of a microsimulation is a truthful reproduction of reality. The required variety of validation data to implement such a concept is just becoming available due to the advancement of information and communication technology. Thus, we are at the right moment of time to animate the discussion about a new validation standard. In addition, research on spatio-temporal data mining and analysis has made tremendous advances during the past decade. As a result, a rich set of preprocessing, feature extraction, indexing and data mining methods are available to exploit and handle spatio-temporal data.

The validation of microsimulation systems is a manifold task. In general, the validation workflow of the modeling process can be divided into three parts, namely input data validation, internal model validation and external model validation (see Figure 1). Input data validation ensures that the model is build using high-quality data. Usually, the provided input data comes from secondary data sources and has originally been collected for a different purpose. For this reason it is important to show in a first validation step that the input data is an appropriate source for the goal of the overall modeling process. Internal model validation measures how well the model can predict its input data, i.e. how reliable a predictive model will perform on (unseen) training data. A high internal quality is a prerequisite for the external quality of the model. If the input data of the model is difficult to predict, this is likely to be the case also for unknown data. In addition, as microsimulation systems typically rely on non-deterministic algorithms, the variability of the model is subject to internal model validation. Finally, during external model validation the model output is compared to either independent real world data or other model results with known high quality. This step ensures the final quality of the model results. If the model outcome meets a given quality standard the process of modeling is finished. Otherwise, either the input data or the modeling process are subject to change and to a repeated validation, leading to a validation cycle.

Figure 1.

Quality cycle in microsimulation systems


Complete Chapter List

Search this Book: