Using Active Rules to Maintain Data Consistency in Data Warehouse Systems

Using Active Rules to Maintain Data Consistency in Data Warehouse Systems

Shi-Ming Huang (National Chung Cheng University, Taiwan), John Tait (Information Retrieval Faculty, Austria), Chun-Hao Su (National Chung Cheng University, Taiwan) and Chih-Fong Tsai (National Central University, Taiwan)
DOI: 10.4018/978-1-60566-232-9.ch012
OnDemand PDF Download:
$37.50

Abstract

Data warehousing is a popular technology, which aims at improving decision-making ability. As the result of an increasingly competitive environment, many companies are adopting a “bottom-up” approach to construct a data warehouse, since it is more likely to be on time and within budget. However, multiple independent data marts/cubes can easily cause problematic data inconsistency for anomalous update transactions, which leads to biased decision-making. This research focuses on solving the data inconsistency problem and proposing a temporal-based data consistency mechanism (TDCM) to maintain data consistency. From a relative time perspective, we use an active rule (standard ECA rule) to monitor the user query event and use a metadata approach to record related information. This both builds relationships between the different data cubes, and allows a user to define a VIT (valid interval temporal) threshold to identify the validity of interval that is a threshold to maintain data consistency. Moreover, we propose a consistency update method to update inconsistent data cubes, which can ensure all pieces of information are temporally consistent.
Chapter Preview
Top

Introduction

Background

Designing and constructing a data warehouse for an enterprise is a very complicated and iterative process since it involves aggregation of data from many different departments and extract, transform, load (ETL) processing (Bellatreche et al., 2001). Currently, there are two basic strategies to implementing a data warehouse, “top-down” and “bottom-up” (Shin, 2002), each with its own strengths, weaknesses, and using the appropriate uses.

Constructing a data warehouse system using the bottom-up approach will be more likely to be on time and within budget. But inconsistent and irreconcilable results may be transmitted from one data mart to the next due to independent data marts or data cubes (e.g. distinct updates time for each data cube) (Inmon, 1998). Thus, inconsistent data in the recognition of events may require a number of further considerations to be taken into account (Shin, 2002; Bruckner et. al, 2001; Song & Liu, 1995):

  • · Data Availability: Typical update patterns for a traditional data warehouse on weekly or even monthly basis will delay discovery, so information is unavailable for knowledge workers or decision makers.

  • · Data Comparability: In order to analyze from different perspectives, or even go a step further to look for more specific information, data comparability is an important issue .

Real-time updating in a data warehouse might be a solution which can enable data warehouses to react “just-in-time” and also provide the best consistency (Bruckner et al., 2001) (e.g. real-time data warehouse). But, not everyone needs or can benefit from a real-time data warehouse. In fact, it is highly possible that only a relatively small portion of the business community will realize a justifiable ROI (return on investment) from a real time data warehouse (Vandermay J., 2001). Real-time data warehouses are expensive to build, requiring a significantly higher level of support and significantly greater investment in infrastructure than a traditional data warehouse. In additional, real-time update will also require high time cost for response and huge storage space for aggregation.

As a result, it is desirable to find an alternative solution for data consistency in a data warehouse system (DWS) which can achieve near real-time outcome but does not require a high cost.

Top

Motivation And Objective

Integrating active rules and data warehouse systems has been one of the most important treads in data warehousing (DM Review, 2001). Active rules have also been used in databases for several years (Paton & Daz, 1999; Roddick & Schrefl, 2000), and much research has been done in this field. It is possible to construct relations between different data cubes or even the data marts. However, anomalous updates could occur when each of the data marts has its own timestamp for obtaining the same data source. Therefore, problems with controlling data consistency in data marts/data cubes are raised.

There have been numerous studies discussing the maintenance of data cubes dealing with the space problem and retrieval efficiency, either by pre-computing a subset of the “possible group-bys” (Harinarayan et al., 1996; Gupta et al., 1997; Baralis et al., 1997), estimating the values of the group-bys using approximation (Gibbons & Matias, 1998; Acharya et al., 2000) or by using online aggregation techniques (Hellerstein et al., 1997; Gray et al., 1996). However, these solutions still focus on single data cube consistency, not on the overall data warehouse environment’s respective. Thus, each department in the enterprise will still face problems of temporal inconsistency over time.

Complete Chapter List

Search this Book:
Reset
Editorial Advisory Board
Table of Contents
Preface
David Taniar
Chapter 1
Laila Niedrite, Maris Solodovnikova Treimanis, Liga Grundmane
There are many methods in the area of data warehousing to define requirements for the development of the most appropriate conceptual model of a data... Sample PDF
Development of Data Warehouse Conceptual Models: Method Engineering Approach
$37.50
Chapter 2
Stefano Rizzi
In the context of data warehouse design, a basic role is played by conceptual modeling, that provides a higher level of abstraction in describing... Sample PDF
Conceptual Modeling Solutions for the Data Warehouse
$37.50
Chapter 3
Hamid Haidarian Shahri
Entity resolution (also known as duplicate elimination) is an important part of the data cleaning process, especially in data integration and... Sample PDF
A Machine Learning Approach to Data Cleaning in Databases and Data Warehouses
$37.50
Chapter 4
Maurizio Pighin, Lucio Ieronutti
Data Warehouses are increasingly used by commercial organizations to extract, from a huge amount of transactional data, concise information useful... Sample PDF
Interactive Quality-Oriented Data Warehouse Development
$37.50
Chapter 5
Dirk Draheim, Oscar Mangisengi
Nowadays tracking data from activity checkpoints of unit transactions within an organization’s business processes becomes an important data resource... Sample PDF
Integrated Business and Production Process Data Warehousing
$37.50
Chapter 6
Jorge Loureiro, Orlando Belo
OLAP queries are characterized by short answering times. Materialized cube views, a pre-aggregation and storage of group-by values, are one of the... Sample PDF
Selecting and Allocating Cubes in Multi-Node OLAP Systems: An Evolutionary Approach
$37.50
Chapter 7
Jorge Loureiro, Orlando Belo
Globalization and market deregulation has increased business competition, which imposed OLAP data and technologies as one of the great enterprise’s... Sample PDF
Swarm Quant' Intelligence for Optimizing Multi-Node OLAP Systems
$37.50
Chapter 8
Franck Ravat, Olivier Teste, Ronan Tournier
With the emergence of Semi-structured data format (such as XML), the storage of documents in centralised facilities appeared as a natural adaptation... Sample PDF
Multidimensional Anlaysis of XML Document Contents with OLAP Dimensions
$37.50
Chapter 9
Hanene Ben-Abdallah, Jamel Feki, Mounira Ben Abdallah
Despite their strategic importance, the wide-spread usage of decision support systems remains limited by both the complexity of their design and the... Sample PDF
A Multidimensional Pattern Based Approach for the Design of Data Marts
$37.50
Chapter 10
Concepción M. Gascueña, Rafael Guadalupe
The Multidimensional Databases (MDB) are used in the Decision Support Systems (DSS) and in Geographic Information Systems (GIS); the latter locates... Sample PDF
A Multidimensional Methodology with Support for Spatio-Temporal Multigranularity in the Conceptual and Logical Phases
$37.50
Chapter 11
Francisco Araque, Alberto Salguero, Cecilia Delgado
One of the most complex issues of the integration and transformation interface is the case where there are multiple sources for a single data... Sample PDF
Methodology for Improving Data Warehouse Design using Data Sources Temporal Metadata
$37.50
Chapter 12
Shi-Ming Huang, John Tait, Chun-Hao Su, Chih-Fong Tsai
Data warehousing is a popular technology, which aims at improving decision-making ability. As the result of an increasingly competitive environment... Sample PDF
Using Active Rules to Maintain Data Consistency in Data Warehouse Systems
$37.50
Chapter 13
Marcin Gorawski, Wojciech Gebczyk
This chapter describes realization of distributed approach to continuous queries with kNN join processing in the spatial telemetric data warehouse.... Sample PDF
Distributed Approach to Continuous Queries with kNN Join Processing in Spatial Telemetric Data Warehouse
$37.50
Chapter 14
Maria Luisa Damiani, Stefano Spaccapietra
This chapter is concerned with multidimensional data models for spatial data warehouses. Over the last few years different approaches have been... Sample PDF
Spatial Data Warehouse Modelling
$37.50
Chapter 15
Jérôme Darmont
Performance evaluation is a key issue for designers and users of Database Management Systems (DBMSs). Performance is generally assessed with... Sample PDF
Data Warehouse Benchmarking with DWEB
$37.50
Chapter 16
Lars Frank, Christian Frank
A Star Schema Data Warehouse looks like a star with a central, so-called fact table, in the middle, surrounded by so-called dimension tables with... Sample PDF
Analyses and Evaluation of Responses to Slowly Changing Dimensions in Data Warehouses
$37.50
About the Contributors