Creating a Data Lakehouse for a South African Government-Sector Learning Control Enforcing Quality Control for Incremental Extract-Load-Transform Pipe

Key Terms in this Chapter

Data Ingestion: The process of collecting and importing information from a variety of sources into an archive, e.g. the Data Lakehouse, so that it can be made available for analysis.

Data Lakehouse: A unified data management architecture that enables organizations to store and analyze structured and unstructured data on a single platform, combining the benefits of Data Lakes and Data Warehouses.

Semi-Structured Data: Data which has an irregular and implicit structure that lacks a defined data model.

Data Orchestration: This is to ensure efficient data flows and integration into a system, often facilitated through automated working processes, by coordinating and managing different processing tasks.

Data Quality: The degree to which data are accurate, complete, timely, and consistent, providing reliable and meaningful information for decision making.