A system capable of dealing with high volume, velocity and variety of data, integrating data from heterogeneous data sources and allowing the extraction of information relevant to the decision-making process. With state-of-art technologies, lower cost solutions can be implemented, overcoming some of the limitations of traditional solutions.
Published in Chapter:
Real-Time Big Data Warehousing
Francisca Vale Lima (University of Minho, Portugal), Carlos Costa (University of Minho, Portugal), and Maribel Yasmina Santos (University of Minho, Portugal)
Copyright: © 2019
|Pages: 30
DOI: 10.4018/978-1-5225-5516-2.ch002
Abstract
The large volume of data that is constantly being generated leads to the need of extracting useful patterns, trends, or insights from this data, raising the interest in business intelligence and big data analytics. The volume, velocity, and variety of data highlight the need for concepts like real-time big data warehouses (RTBDWs). The lack of guidelines or methodological approaches for implementing these systems requires further research in this recent topic. This chapter presents the proposal of a RTBDW architecture that includes the main components and data flows needed to collect, process, store, and analyze the available data, integrating streaming with batch data and enabling real-time decision making. Using Twitter data, several technologies were evaluated to understand their performance. The obtained results were satisfactory and allowed the identification of a methodological approach that can be followed for the implementation of this type of system.