Quality Metrics for Evaluating Data Provenance

Introduction

Over the last decade, data management issues have grown in complexity owning to huge amounts of data being generated in various areas of computational sciences such as biology, meteorology, physics, astronomy, chemistry, geophysics, and weather forecasting. This has been enabled by sophisticated methods of experiments and observations in the wet labs, enabled through an ever advancing hardware technology and also through simulations in dry lab environment (Buneman, Khanna, & Tan, 2001). These data repositories have been made available to the researchers, scientists and other consumers of this data through the Internet and Web-based applications. With the growing importance of the data intensive domains such as Bioinformatics, Geographical Information System (GIS) and Weather Forecasting, reliance on these data driven scientific Web applications has increased. The availability of these valuable assets of data has contributed immensely to the advancement of research and toward scientific discovery. However, an increased reliance on exchange and sharing of these data resources have put the notion of data quality at the forefront of research among the database community as concerns about the quality of data have been amplified (Jagadish & Olken, 2004). These applications execute complex workflows, consuming and producing data products of unknown quality. This poses serious challenges of selecting “right and correct” data for the scientists using these data products for various purposes such as insilico experiments and computational investigations.

Data Provenance is a kind of metadata which tracks information about the data such as its evolution, including source and authority, creation, life history including its usage history and other information about the agents of change (Wadhwa & Kamalapur, 2003). It provides a qualitative and quantitative metrics to analyze the quality and the dependability of the data. Hence, data provenance is valuable parameter to scientists working in domains such as Bioinformatics, where the quality and sources of the underlying data could play a major role in the quality of their experiments and research.

In the research, a few data quality models have been proposed to verify the quality of data that is being used (Buneman & Khanna, 2001; Simmhan et al., 2005; Ismael Caballero & Piattini, 2003). In these models, data provenance has been proposed as one of the many quality metrics to determine the quality of data. But to the best of our knowledge, so far data provenance itself has not been investigated in detail. In our opinion, data provenance is a critical and important indicator for determining and measuring the data quality and it needs further research.

In this chapter, we first explore the current understanding of the data provenance, its categorization based on the domains and where it can be applied. Then, we identify different applications of data provenance in various domains, and finally propose a set of parameters and related metrics to measure the data provenance. These proposed metrics can assist the researchers to better understand data provenance and design provenance systems.

The remainder of the book chapter is organized as follows. In the second section, we give literature survey of the field. The issues and problems of data provinces are identified and discussed in the next section. Then, we suggest a set of metrics for the measuring data provenance. Finally, we give concluding remarks and future directions of this work.