Swarm Quant' Intelligence for Optimizing Multi-Node OLAP Systems

Swarm Quant' Intelligence for Optimizing Multi-Node OLAP Systems

Jorge Loureiro (Instituto Politécnico de Viseu, Portugal) and Orlando Belo (Universidade do Minho, Portugal)
DOI: 10.4018/978-1-60566-232-9.ch007
OnDemand PDF Download:
$37.50

Abstract

Globalization and market deregulation has increased business competition, which imposed OLAP data and technologies as one of the great enterprise’s assets. Its growing use and size stressed underlying servers and forced new solutions. The distribution of multidimensional data through a number of servers allows the increasing of storage and processing power without an exponential increase of financial costs. However, this solution adds another dimension to the problem: space. Even in centralized OLAP, cube selection efficiency is complex, but now, we must also know where to materialize subcubes. We have to select and also allocate the most beneficial subcubes, attending an expected (changing) user profile and constraints. We now have to deal with materializing space, processing power distribution, and communication costs. This chapter proposes new distributed cube selection algorithms based on discrete particle swarm optimizers; algorithms that solve the distributed OLAP selection problem considering a query profile under space constraints, using discrete particle swarm optimization in its normal(Di-PSO), cooperative (Di-CPSO), multi-phase (Di-MPSO), and applying hybrid genetic operators.
Chapter Preview
Top

Introduction

Nowadays, economy, with globalization (market opening and unrulement) shows a growing dynamic and volatile environment. Decision makers submerged into uncertainty are eager for something to guide them, in order to make timely, coherent and adjusted decisions. Within this context a new Grail was born: information as condition for competition. Data Warehouses (DW) emerged, naturally, as a core component in the constitution of organization’s informational infrastructure. Their unified, subject oriented, non-volatile and temporal variability preserving vision, allowed them to become the main source of information concerning business activities.

The growing interest on DW’s information by knowledge workers has motivated a fast enlargement of the business enclosed areas. Also the adoption of Data Warehousing (DWing) by most of Fortune 400’s enterprises has helped to make the huge size of today’s DW (hundreds of GB or even tenths of TB). A query addressed to such a database has necessarily a long run-time, but they must be desirably short, given the on-line appanage characteristic of OLAP systems. This emphasis on speed is dictated by two orders of reasons: 1) OLAP users’ need to take business decisions in a few minutes, in order to accompany the fast change of markets, operated in short time intervals; 2) the strong dependence of the productivity of CEO’s, managers and all knowledge workers and decision makers of enterprises in general, on the quickness of the answers to their business questions.

However, this constant need for speed seems to be blocked by the huge amount of DW data: a query like “show me the sales by product family and month of this year related to last year” may force a scanning and aggregation of a significant portion of the fact table in the DW. This is something that could last for hours or days, even disposing, hypothetically, of powerful hardware and suitable indexes.

The adoption of a DWing “eagger” (Widom, 1995) approach allowed to solve this problem through the generation and timely updating of the so called materialized views, summary tables or subcubes (mainly used from now on). In essence, they are Group By previously calculated and stored by any kind of dimensions/hierarchies’ combinations. These subcubes need space and especially time, enlarging the size of the DW even more, perhaps one hundred times bigger, since the number of subcubes may be very large, causing the well-known “data explosion”. So, it is crucial to restrict the number of subcubes and select those that prove to be the most useful, due to their ratio utilization/occupied space. This is, in the essence, the views selection problem: selecting the right set of subcubes to materialize, in order to minimize query costs, characteristically NP-hard (Harinarayan, Rajaraman, and Ullman, 1996).

Two constraints may be applied to the optimization process: the space that is available to cube materializing and the time disposable to the refreshing process. But multidimensional data continues growing and the number of OLAP users too. These concomitant factors impose a great stress over OLAP underlying platform: a new powerful server was needed or simply empower the architecture with the aggregated power of several small (general purpose) servers, distributing multidimensional structures and OLAP queries through the available nodes. That’s what we called Multi-Node OLAP architecture, shown in Figure 1.

Figure 1.

Multi-Node OLAP (M-OLAP) Architecture and corresponding framing (data sources, DW, ETLs processes, and administration and restructuring engine).

A number of OLAP Server Nodes (OSN) with predefined storage and processing power, connected through a network, using real characteristics of communication inter-node links, which may freely share data or issue aggregation queries to other nodes participating in a distributed scenario, constitutes the M-OLAP component, where inhabit the multidimensional structures. This system serves a distributed knowledge worker community, which puts a set of queries on their daily routine. This brings to OLAP the known advantages of data distribution and processing proficiently, like increased availability, communication costs reduction, simpler and cheaper hardware and loading and processing distribution.

Complete Chapter List

Search this Book:
Reset
Editorial Advisory Board
Table of Contents
Preface
David Taniar
Chapter 1
Laila Niedrite, Maris Solodovnikova Treimanis, Liga Grundmane
There are many methods in the area of data warehousing to define requirements for the development of the most appropriate conceptual model of a data... Sample PDF
Development of Data Warehouse Conceptual Models: Method Engineering Approach
$37.50
Chapter 2
Stefano Rizzi
In the context of data warehouse design, a basic role is played by conceptual modeling, that provides a higher level of abstraction in describing... Sample PDF
Conceptual Modeling Solutions for the Data Warehouse
$37.50
Chapter 3
Hamid Haidarian Shahri
Entity resolution (also known as duplicate elimination) is an important part of the data cleaning process, especially in data integration and... Sample PDF
A Machine Learning Approach to Data Cleaning in Databases and Data Warehouses
$37.50
Chapter 4
Maurizio Pighin, Lucio Ieronutti
Data Warehouses are increasingly used by commercial organizations to extract, from a huge amount of transactional data, concise information useful... Sample PDF
Interactive Quality-Oriented Data Warehouse Development
$37.50
Chapter 5
Dirk Draheim, Oscar Mangisengi
Nowadays tracking data from activity checkpoints of unit transactions within an organization’s business processes becomes an important data resource... Sample PDF
Integrated Business and Production Process Data Warehousing
$37.50
Chapter 6
Jorge Loureiro, Orlando Belo
OLAP queries are characterized by short answering times. Materialized cube views, a pre-aggregation and storage of group-by values, are one of the... Sample PDF
Selecting and Allocating Cubes in Multi-Node OLAP Systems: An Evolutionary Approach
$37.50
Chapter 7
Jorge Loureiro, Orlando Belo
Globalization and market deregulation has increased business competition, which imposed OLAP data and technologies as one of the great enterprise’s... Sample PDF
Swarm Quant' Intelligence for Optimizing Multi-Node OLAP Systems
$37.50
Chapter 8
Franck Ravat, Olivier Teste, Ronan Tournier
With the emergence of Semi-structured data format (such as XML), the storage of documents in centralised facilities appeared as a natural adaptation... Sample PDF
Multidimensional Anlaysis of XML Document Contents with OLAP Dimensions
$37.50
Chapter 9
Hanene Ben-Abdallah, Jamel Feki, Mounira Ben Abdallah
Despite their strategic importance, the wide-spread usage of decision support systems remains limited by both the complexity of their design and the... Sample PDF
A Multidimensional Pattern Based Approach for the Design of Data Marts
$37.50
Chapter 10
Concepción M. Gascueña, Rafael Guadalupe
The Multidimensional Databases (MDB) are used in the Decision Support Systems (DSS) and in Geographic Information Systems (GIS); the latter locates... Sample PDF
A Multidimensional Methodology with Support for Spatio-Temporal Multigranularity in the Conceptual and Logical Phases
$37.50
Chapter 11
Francisco Araque, Alberto Salguero, Cecilia Delgado
One of the most complex issues of the integration and transformation interface is the case where there are multiple sources for a single data... Sample PDF
Methodology for Improving Data Warehouse Design using Data Sources Temporal Metadata
$37.50
Chapter 12
Shi-Ming Huang, John Tait, Chun-Hao Su, Chih-Fong Tsai
Data warehousing is a popular technology, which aims at improving decision-making ability. As the result of an increasingly competitive environment... Sample PDF
Using Active Rules to Maintain Data Consistency in Data Warehouse Systems
$37.50
Chapter 13
Marcin Gorawski, Wojciech Gebczyk
This chapter describes realization of distributed approach to continuous queries with kNN join processing in the spatial telemetric data warehouse.... Sample PDF
Distributed Approach to Continuous Queries with kNN Join Processing in Spatial Telemetric Data Warehouse
$37.50
Chapter 14
Maria Luisa Damiani, Stefano Spaccapietra
This chapter is concerned with multidimensional data models for spatial data warehouses. Over the last few years different approaches have been... Sample PDF
Spatial Data Warehouse Modelling
$37.50
Chapter 15
Jérôme Darmont
Performance evaluation is a key issue for designers and users of Database Management Systems (DBMSs). Performance is generally assessed with... Sample PDF
Data Warehouse Benchmarking with DWEB
$37.50
Chapter 16
Lars Frank, Christian Frank
A Star Schema Data Warehouse looks like a star with a central, so-called fact table, in the middle, surrounded by so-called dimension tables with... Sample PDF
Analyses and Evaluation of Responses to Slowly Changing Dimensions in Data Warehouses
$37.50
About the Contributors