SQLBits 2024
End-to-end data validation strategies in Microsoft Fabric
A strategic look at the WHY, WHEN, and HOW of validating your data through each stage of a medallion-style architecture in Microsoft Fabric. Your end-users will be glad you came to this talk.
Without validating your data, could it be argued that all visualisations you make or analyses you do with this data are useless? Or perhaps worse: unknowingly misleading?
Business users expect data to be correct, and that is a fair expectation. In practice, any data practitioner will know that this can be a difficult expectation to meet consistently. The answer is robust data validation, and Microsoft Fabric makes this task easier than ever.
At the end of this session, you will leave with an appreciation for the importance of good data validation (WHY).
Together we will navigate an end-to-end data processing pipeline in Microsoft Fabric and discuss strategies for data validation at each stage: from ingestion at source, through layers of a medallion-like architecture, to validation of semantic modelling (including measures) in Power BI.
Keywords for this session: Data Validation, Data Strategy, Microsoft Fabric, Medallion Architecture, Synapse Lakehouse, Synapse Data Warehouse, Great Expectations, Semantic Link.
Business users expect data to be correct, and that is a fair expectation. In practice, any data practitioner will know that this can be a difficult expectation to meet consistently. The answer is robust data validation, and Microsoft Fabric makes this task easier than ever.
At the end of this session, you will leave with an appreciation for the importance of good data validation (WHY).
Together we will navigate an end-to-end data processing pipeline in Microsoft Fabric and discuss strategies for data validation at each stage: from ingestion at source, through layers of a medallion-like architecture, to validation of semantic modelling (including measures) in Power BI.
Keywords for this session: Data Validation, Data Strategy, Microsoft Fabric, Medallion Architecture, Synapse Lakehouse, Synapse Data Warehouse, Great Expectations, Semantic Link.