#lakehouse

Articles tagged with #lakehouse

Delta Lake schema enforcement and evolution: the guardrail and the trap
Schema enforcement is the Delta Lake feature that quietly saves you, rejecting the write that would have corrupted a table. Schema evolution is the one that bites you, because teams reach for it to make an error go away and end up silently
Jul 12, 20267 min read
Building a Medallion Lakehouse on AWS Glue and Iceberg (No Databricks)
We built a production medallion lakehouse for a regulated credit bureau using only AWS native services: Glue, Iceberg, MWAA, RDS, and Athena. Here is the full end to end stack, why each layer exists, and the honest tradeoffs of skipping Dat
Jul 12, 202610 min read
SCD Type 4 in a Lakehouse: History and Field Audit from Iceberg Snapshots
Keeping a Gold dimension clean while still answering "who changed what and when" means splitting current state from history. Here is how we drive a slowly changing dimension type 4 pattern off Apache Iceberg snapshot diffing, with separate
Jul 12, 202610 min read
Validating Data Against Its Own History: Baseline Data Quality with GlueDQ
Row level checks catch nulls and bad types, but they wave through data that is well formed yet wrong, like a contributor whose arrears drop 90 percent in a month. The fix is a Hybrid Baseline check that validates each batch against its own
Jul 12, 20268 min read