Quality Metrics for Evaluating Data Provenance

Quality Metrics for Evaluating Data Provenance

Syed Ahsan (University of Engineering and Technology, Pakistan) and Abad Shah (University of Engineering and Technology, Pakistan)
Copyright: © 2009 |Pages: 19
DOI: 10.4018/978-1-59904-699-0.ch015
OnDemand PDF Download:


With the proliferation of Web, a tremendous amount of data is available to researchers and scientists in computational sciences, business organizations and general public. This has resulted in an increased importance of data intensive domains such as Bioinformatics, which are increasingly using Web-based applications and service-oriented architecture which uses the data available on the Web sources. To trust the data available on Web and the results derived from it, a Data Provenance system must be devised to ensure authenticity and credibility of Web resources. In this paper we have discussed various domains which necessitate such data provenance systems. We propose a set of tangible parameters which affect the quality of data and define quality metrics to evaluate those parameters. The chapter concludes with a section on future directions in which we identify various research problems and possible applications of data provenance.
Chapter Preview


Over the last decade, data management issues have grown in complexity owning to huge amounts of data being generated in various areas of computational sciences such as biology, meteorology, physics, astronomy, chemistry, geophysics, and weather forecasting. This has been enabled by sophisticated methods of experiments and observations in the wet labs, enabled through an ever advancing hardware technology and also through simulations in dry lab environment (Buneman, Khanna, & Tan, 2001). These data repositories have been made available to the researchers, scientists and other consumers of this data through the Internet and Web-based applications. With the growing importance of the data intensive domains such as Bioinformatics, Geographical Information System (GIS) and Weather Forecasting, reliance on these data driven scientific Web applications has increased. The availability of these valuable assets of data has contributed immensely to the advancement of research and toward scientific discovery. However, an increased reliance on exchange and sharing of these data resources have put the notion of data quality at the forefront of research among the database community as concerns about the quality of data have been amplified (Jagadish & Olken, 2004). These applications execute complex workflows, consuming and producing data products of unknown quality. This poses serious challenges of selecting “right and correct” data for the scientists using these data products for various purposes such as insilico experiments and computational investigations.

Data Provenance is a kind of metadata which tracks information about the data such as its evolution, including source and authority, creation, life history including its usage history and other information about the agents of change (Wadhwa & Kamalapur, 2003). It provides a qualitative and quantitative metrics to analyze the quality and the dependability of the data. Hence, data provenance is valuable parameter to scientists working in domains such as Bioinformatics, where the quality and sources of the underlying data could play a major role in the quality of their experiments and research.

In the research, a few data quality models have been proposed to verify the quality of data that is being used (Buneman & Khanna, 2001; Simmhan et al., 2005; Ismael Caballero & Piattini, 2003). In these models, data provenance has been proposed as one of the many quality metrics to determine the quality of data. But to the best of our knowledge, so far data provenance itself has not been investigated in detail. In our opinion, data provenance is a critical and important indicator for determining and measuring the data quality and it needs further research.

In this chapter, we first explore the current understanding of the data provenance, its categorization based on the domains and where it can be applied. Then, we identify different applications of data provenance in various domains, and finally propose a set of parameters and related metrics to measure the data provenance. These proposed metrics can assist the researchers to better understand data provenance and design provenance systems.

The remainder of the book chapter is organized as follows. In the second section, we give literature survey of the field. The issues and problems of data provinces are identified and discussed in the next section. Then, we suggest a set of metrics for the measuring data provenance. Finally, we give concluding remarks and future directions of this work.

Complete Chapter List

Search this Book:
Table of Contents
Pierre F. Tiako
Chapter 1
Alf Inge Wang, Carl-Fredrik Sørensen
This chapter presents a framework for differentiated process support in large software projects. Process support can be differentiated in different... Sample PDF
Differentiated Process Support for Large Software Projects
Chapter 2
Holger Giese, Stefan Henkler, Martin Hirsch, Vladimir Rubin, Matthias Tichy
Software has become the driving force in the evolution of many systems, such as embedded systems (especially automotive applications)... Sample PDF
Modeling Techniques for Software-Intensive Systems
Chapter 3
Jaroslav Král, Michal Žemlicka
Software intensive systems are systems strongly depending on supporting software. The software is typically large, complex, and it as a rule... Sample PDF
Service Orientation and Software-Intensive Systems
Chapter 4
Alf Inge Wang, Carl-Fredrik Sørensen, Hien Nam Le, Heri Ramampiaro, Mads Nygård, Reidar Conradi
This chapter describes a requirement analysis framework that may be used as a tool for developing client-server systems for mobile workers. The... Sample PDF
From Scenarios to Requirements in Mobile Client-Server Systems
Chapter 5
Gan Deng, Douglas C. Schmidt, Aniruddha Gokhale, Jeff Gray, Yuehua Lin, Gunther Lenz
This chapter describes our approach to model-driven engineering (MDE)-based product line architectures (PLAs) and presents a solution to address the... Sample PDF
Evolution in Model-Driven Software Product-Line Architectures
Chapter 6
Ståle Walderhaug, Erlend Stav, Ulrik Johansen, Gøran K. Olsen
This chapter introduces a tracability solution for use in a model-driven software development approach. It argues that a trace model based on a... Sample PDF
Traceability in Model-Driven Software Development
Chapter 7
Gerhard Chroust, Erwin Schoitsch
When designing a complex software-intensive system it is unavoidable to make some a-priori basic assumptions about its architecture. We introduce... Sample PDF
Choosing Basic Architectural Alternatives
Chapter 8
Rafael Capilla, Margarita Martínez, Francisco Nava, Cristina Muñoz
Virtual reality systems are a kind of complex software systems that need a lot of effort and resources during its development phase. Because rigid... Sample PDF
Architecting Virtual Reality Systems
Chapter 9
Kendra M.L. Cooper, Lirong Dai, Renee Steiner, Rym Zalila Mili
This chapter presents a survey of software architecture approaches. It is intended for a broad audience of students, practitioners, and researchers... Sample PDF
A Survey of Software Architecture Approaches
Chapter 10
Daniel G. Waddington, Nilabja Roy, Douglas C. Schmidt
As software-intensive systems become larger, more parallel, and more unpredictable the ability to analyze their behavior is increasingly important.... Sample PDF
Dynamic Analysis and Profiling of Multithreaded Systems
Chapter 11
James H. Hill, Douglas C. Schmidt, John M. Slaby
This chapter introduces the next generation of system execution modeling tools designed around modeldriven engineering (MDE) coupled with... Sample PDF
Evaluating Quality of Service for Enterprise Distributed Systems
Chapter 12
Jules White, Douglas C. Schmidt, Andrey Nechypurenko, Egon Wuchner
Model-driven development is one approach to combating the complexity of designing software intensive systems. A model-driven approach allows... Sample PDF
Reducing the Complexity of Modeling Large Software Systems
Chapter 13
Enis Afgan, Purushotham Bangalore, Jeff Gray
Grid computing environments are dynamic and heterogeneous in nature. In order to realize applicationspecific Quality of Service agreements within a... Sample PDF
A Domain-Specific Language for Describing Grid Applications
Chapter 14
Jeff Elpern, Sergiu Dascalu
Traditional software engineering methodologies have mostly evolved from the environment of proprietary, large-scale software systems. Here, software... Sample PDF
A Framework for Understanding the Open Source Revolution
Chapter 15
Syed Ahsan, Abad Shah
With the proliferation of Web, a tremendous amount of data is available to researchers and scientists in computational sciences, business... Sample PDF
Quality Metrics for Evaluating Data Provenance
Chapter 16
Krishnakumar Balasubramanian, Douglas C. Schmidt, Zoltán Molnár, Ákos Lédeczi
With the emergence of commercial-off-the-shelf (COTS) component middleware technologies software system integrators are increasing faced with the... Sample PDF
System Integration Using Model-Driven Engineering
About the Contributors