Extraction, Transformation, and Loading Processes

Abstract

ETL stands for Extraction, Transformation, and Loading: in other words, for the Data Warehouse (DW) backstage. The main focus of our exposition here is the practical application of the ETL process in real world cases with extra problems and strong requirements, particularly performance issues related to population of large Data Warehouse. In a context of ETL/DW with strong requirements, we can individuate the most common constraints and criticalities that one can meet in developing an ETL system. We will describe some techniques related to the physical data base design, pipelining, and parallelism which are crucial for the whole ETL process. We will propose our practical approach “infrastructure based ETL”: it is not a tool but a set of functionalities and/or services that the experience has proved to be useful and enough widespread in the ETL scenario and one can build the application on top of it.