My Data Is Good?

My Data Is Good?

In this “shortening soup” session, we’ll dive into the classic GIGO (Garbage In, Garbage Out) challenge when building ETL pipelines – whether in Databricks, Fabric, or elsewhere. How do you validate your data before it flows downstream? What are the “expectations” (pun intended) for data quality, and how can you ensure your medallion architecture shines all the way to gold?

We’ll unwrap the tools and techniques available for data validation and cleaning at each stage – bronze, silver, and gold – exploring when to use which tool and the benefits each brings. By the end, you’ll have a clear, practical approach to data validation, so you won’t be left scrambling when you realize (and you will!) just how essential it is. And yes, there will be sweetness all along the way, Kinder chocolate sweetness!