CDC into a lakehouse: Change Data Feed, MERGE, and not reprocessing everything
The moment a pipeline has to reflect updates and deletes from a source, not just appends, full reloads stop scaling and append-only logic starts lying. Change data capture done properly means capturing what changed, applying it with MERGE,
Jul 12, 20267 min read
