Incremental Load in a Data Warehousing Environment

Incremental Load in a Data Warehousing Environment

Nayem Rahman
ISBN13: 9781466601581|ISBN10: 1466601582|EISBN13: 9781466601598
DOI: 10.4018/978-1-4666-0158-1.ch009
Cite Chapter Cite Chapter

MLA

Rahman, Nayem. "Incremental Load in a Data Warehousing Environment." Insights into Advancements in Intelligent Information Technologies: Discoveries, edited by Vijayan Sugumaran, IGI Global, 2012, pp. 161-177. https://doi.org/10.4018/978-1-4666-0158-1.ch009

APA

Rahman, N. (2012). Incremental Load in a Data Warehousing Environment. In V. Sugumaran (Ed.), Insights into Advancements in Intelligent Information Technologies: Discoveries (pp. 161-177). IGI Global. https://doi.org/10.4018/978-1-4666-0158-1.ch009

Chicago

Rahman, Nayem. "Incremental Load in a Data Warehousing Environment." In Insights into Advancements in Intelligent Information Technologies: Discoveries, edited by Vijayan Sugumaran, 161-177. Hershey, PA: IGI Global, 2012. https://doi.org/10.4018/978-1-4666-0158-1.ch009

Export Reference

Mendeley
Favorite

Abstract

Incremental load is an important factor for successful data warehousing. Lack of standardized incremental refresh methodologies can lead to poor analytical results, which can be unacceptable to an organization’s analytical community. Successful data warehouse implementation depends on consistent metadata as well as incremental data load techniques. If consistent load timestamps are maintained and efficient transformation algorithms are used, it is possible to refresh databases with complete accuracy and with little or no manual checking. This paper proposes an Extract-Transform-Load (ETL) metadata model that archives load observation timestamps and other useful load parameters. The author also recommends algorithms and techniques for incremental refreshes that enable table loading while ensuring data consistency, integrity, and improving load performance. In addition to significantly improving quality in incremental load techniques, these methods will save a substantial amount of data warehouse systems resources.

Request Access

You do not own this content. Please login to recommend this title to your institution's librarian or purchase it from the IGI Global bookstore.