Interactive Quality-Oriented Data Warehouse Development

Interactive Quality-Oriented Data Warehouse Development

Maurizio Pighin (IS&SE-Lab and University of Udine, Italy) and Lucio Ieronutti (IS&SE-Lab and University of Udine, Italy)
DOI: 10.4018/978-1-60566-232-9.ch004
OnDemand PDF Download:
$37.50

Abstract

Data Warehouses are increasingly used by commercial organizations to extract, from a huge amount of transactional data, concise information useful for supporting decision processes. However, the task of designing a data warehouse and evaluating its effectiveness is not trivial, especially in the case of large databases and in presence of redundant information. The meaning and the quality of selected attributes heavily influence the data warehouse’s effectiveness and the quality of derived decisions. Our research is focused on interactive methodologies and techniques targeted at supporting the data warehouse design and evaluation by taking into account the quality of initial data. In this chapter we propose an approach for supporting the data warehouses development and refinement, providing practical examples and demonstrating the effectiveness of our solution. Our approach is mainly based on two phases: the first one is targeted at interactively guiding the attributes selection by providing quantitative information measuring different statistical and syntactical aspects of data, while the second phase, based on a set of 3D visualizations, gives the opportunity of run-time refining taken design choices according to data examination and analysis. For experimenting proposed solutions on real data, we have developed a tool, called ELDA (EvaLuation DAta warehouse quality), that has been used for supporting the data warehouse design and evaluation.
Chapter Preview
Top

Introduction

Data Warehouses are widely used by commercial organizations to extract from an huge amount of transactional data concise information useful for supporting decision processes. For example, organization managers greatly benefit from the availability of tools and techniques targeted at deriving information on sale trends and discovering unusual accounting movements. With respect to the entire amount of data stored into the initial database (or databases, hereinafter DBs), such analysis is centered on a limited subset of attributes (i.e., datawarehouse measures and dimensions). As a result, the datawarehouse (hereinafter DW) effectiveness and the quality of related decision is strongly influenced by the semantics of selected attributes and the quality of initial data. For example, information on customers and suppliers as well as products ordered and sold are very meaningful from data analysis point of view due to their semantics. However, the availability of information measuring and representing different aspect of data can make easier the task of selecting DW attributes, especially in presence of multiple choices (i.e., redundant information) and in the case of DBs characterized by an high number of attributes, tables and relations. Quantitative measurements allows DW engineers to better focus their attention towards the attributes characterized by the most desirable features, while qualitative data representations enables one to interactively and intuitively examine the considered data subset, allowing one to reduce the time required for the DW design and evaluation.

Our research is focused on interactive methodologies and techniques aimed at supporting the DW design and evaluation by taking into account the quality of initial data. In this chapter we propose an approach supporting the DW development and refinement, providing practical examples demonstrating the effectiveness of our solution. Proposed methodology can be effectively used (i) during the DW construction phase for driving and interactively refining the attributes selection, and (ii) at the end of the design process, to evaluate the quality of taken DW design choices.

While most solutions that have been proposed in the literature for assessing data quality are related with semantics, our goal is to propose an interactive approach focused on statistical aspects of data. The approach is mainly composed by two phases: an analytical phase based on a set of metrics measuring different data features (quantitative information), and an exploration phase based on an innovative graphical representation of DW ipercubes that allows one to navigate intuitively through the information space to better examine the quality and data distribution (qualitative information). The interaction is one of the most important feature of our approach: the designer can incrementally defines the DW measures and dimensions and both quality measurements and data representations change according to such modifications. This solution allows one to evaluate rapidly and intuitively the effects of alternative design choices. For example, the designer can immediately discover that the inclusion of an attribute negatively influences the global DW quality. If the quantitative evaluation does not convince the designer, he can explore the DW ipercubes to better understand relations among data, data distributions and behaviors.

In a real world scenario, DW engineers greatly benefit from the possibility of obtaining concise and easy-to-understand information describing the data actually stored into the DB, since they typically have a partial knowledge and vision of a specific operational DB (e.g., how an organization really uses the commercial system). Indeed, different commercial organizations can use the same information system, but each DB instantiation stores data that can be different from the point of view of distribution, correctness and reliability (e.g., an organization never fills a particular field of the form). As a result, the same DW design choices can produce different informative effects depending on the data actually stored into the DB. Then, although the attributes selection is primarily based on data semantics, the availability of both quantitative and qualitative information on data could greatly support the DW design phase. For example, in the presence of alternative choices (valid from semantic point of view), the designer can select the attribute characterized by the most desirable syntactical and statistical features. On the other hand, the designer can decide to change his design choice if he discovers that the selected attribute is characterized by undesirable features (for instance, an high percentage of null values).

Complete Chapter List

Search this Book:
Reset
Editorial Advisory Board
Table of Contents
Preface
David Taniar
Chapter 1
Laila Niedrite, Maris Solodovnikova Treimanis, Liga Grundmane
There are many methods in the area of data warehousing to define requirements for the development of the most appropriate conceptual model of a data... Sample PDF
Development of Data Warehouse Conceptual Models: Method Engineering Approach
$37.50
Chapter 2
Stefano Rizzi
In the context of data warehouse design, a basic role is played by conceptual modeling, that provides a higher level of abstraction in describing... Sample PDF
Conceptual Modeling Solutions for the Data Warehouse
$37.50
Chapter 3
Hamid Haidarian Shahri
Entity resolution (also known as duplicate elimination) is an important part of the data cleaning process, especially in data integration and... Sample PDF
A Machine Learning Approach to Data Cleaning in Databases and Data Warehouses
$37.50
Chapter 4
Maurizio Pighin, Lucio Ieronutti
Data Warehouses are increasingly used by commercial organizations to extract, from a huge amount of transactional data, concise information useful... Sample PDF
Interactive Quality-Oriented Data Warehouse Development
$37.50
Chapter 5
Dirk Draheim, Oscar Mangisengi
Nowadays tracking data from activity checkpoints of unit transactions within an organization’s business processes becomes an important data resource... Sample PDF
Integrated Business and Production Process Data Warehousing
$37.50
Chapter 6
Jorge Loureiro, Orlando Belo
OLAP queries are characterized by short answering times. Materialized cube views, a pre-aggregation and storage of group-by values, are one of the... Sample PDF
Selecting and Allocating Cubes in Multi-Node OLAP Systems: An Evolutionary Approach
$37.50
Chapter 7
Jorge Loureiro, Orlando Belo
Globalization and market deregulation has increased business competition, which imposed OLAP data and technologies as one of the great enterprise’s... Sample PDF
Swarm Quant' Intelligence for Optimizing Multi-Node OLAP Systems
$37.50
Chapter 8
Franck Ravat, Olivier Teste, Ronan Tournier
With the emergence of Semi-structured data format (such as XML), the storage of documents in centralised facilities appeared as a natural adaptation... Sample PDF
Multidimensional Anlaysis of XML Document Contents with OLAP Dimensions
$37.50
Chapter 9
Hanene Ben-Abdallah, Jamel Feki, Mounira Ben Abdallah
Despite their strategic importance, the wide-spread usage of decision support systems remains limited by both the complexity of their design and the... Sample PDF
A Multidimensional Pattern Based Approach for the Design of Data Marts
$37.50
Chapter 10
Concepción M. Gascueña, Rafael Guadalupe
The Multidimensional Databases (MDB) are used in the Decision Support Systems (DSS) and in Geographic Information Systems (GIS); the latter locates... Sample PDF
A Multidimensional Methodology with Support for Spatio-Temporal Multigranularity in the Conceptual and Logical Phases
$37.50
Chapter 11
Francisco Araque, Alberto Salguero, Cecilia Delgado
One of the most complex issues of the integration and transformation interface is the case where there are multiple sources for a single data... Sample PDF
Methodology for Improving Data Warehouse Design using Data Sources Temporal Metadata
$37.50
Chapter 12
Shi-Ming Huang, John Tait, Chun-Hao Su, Chih-Fong Tsai
Data warehousing is a popular technology, which aims at improving decision-making ability. As the result of an increasingly competitive environment... Sample PDF
Using Active Rules to Maintain Data Consistency in Data Warehouse Systems
$37.50
Chapter 13
Marcin Gorawski, Wojciech Gebczyk
This chapter describes realization of distributed approach to continuous queries with kNN join processing in the spatial telemetric data warehouse.... Sample PDF
Distributed Approach to Continuous Queries with kNN Join Processing in Spatial Telemetric Data Warehouse
$37.50
Chapter 14
Maria Luisa Damiani, Stefano Spaccapietra
This chapter is concerned with multidimensional data models for spatial data warehouses. Over the last few years different approaches have been... Sample PDF
Spatial Data Warehouse Modelling
$37.50
Chapter 15
Jérôme Darmont
Performance evaluation is a key issue for designers and users of Database Management Systems (DBMSs). Performance is generally assessed with... Sample PDF
Data Warehouse Benchmarking with DWEB
$37.50
Chapter 16
Lars Frank, Christian Frank
A Star Schema Data Warehouse looks like a star with a central, so-called fact table, in the middle, surrounded by so-called dimension tables with... Sample PDF
Analyses and Evaluation of Responses to Slowly Changing Dimensions in Data Warehouses
$37.50
About the Contributors