Abstract
Data does not stand still. As data warehouse developers, this is a known fact on which our careers are based. For data to have value, it has to be reliably moved to a place where that value can be realized and the method by which we move data should depend on the needs of our users and the frequency of the data, not on the physical or technological limits of the system. As this book examines a modern data warehouse, we need to research beyond the traditional defaults such as batch-based ingestion and simple lift and shift extract, transform, and load (ETL) patterns and explore how we offer more flexibility to the end users. This chapter outlines an approach for warehouse loading that promotes efficiency and resilience, moving on to describe three ingestion modes. By defining the risks and benefits of batch-based, event-based, and streaming modes, you will know how to implement each approach while also being aware of the additional complexities of each, ensuring a successful implementation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2020 Matt How
About this chapter
Cite this chapter
How, M. (2020). The Ingestion Architecture. In: The Modern Data Warehouse in Azure. Apress, Berkeley, CA. https://doi.org/10.1007/978-1-4842-5823-1_4
Download citation
DOI: https://doi.org/10.1007/978-1-4842-5823-1_4
Published:
Publisher Name: Apress, Berkeley, CA
Print ISBN: 978-1-4842-5822-4
Online ISBN: 978-1-4842-5823-1
eBook Packages: Professional and Applied ComputingApress Access BooksProfessional and Applied Computing (R0)