GDPR is a Storage Problem: Implementing Hard Deletes in Apache Iceberg
Data deletion in a lakehouse is a lie if you only run a DELETE statement. In regulated environments,...
Tag archive
Data deletion in a lakehouse is a lie if you only run a DELETE statement. In regulated environments,...

Iceberg v4 is adding column files, companion files that can hold one or more whole columns for every row of a data file and be swapped in by...

We put the scheduled Spark compaction job that most Iceberg teams write on a table taking a MERGE...

Deletion vectors are bound to one data file. Equality deletes are bound to nothing, which is why a CDC-fed Iceberg table gets slower to read every week.
I SIGKILLed a Spark Structured Streaming job writing to Iceberg mid-backlog, then counted offsets. The test, and what it does not prove.
Original Japanese article: AWSから見たSnowflake Storage for Iceberg Introduction I'm Aki,...

Firat Celik's DuckDB 2.0 vs Polars vs Spark on one machine: 150M rows under a 4.5 GiB cap asks a...

Hi Folks, Snowflake AWS → Azure → Microsoft Fabric — Gotchas, Part 4 I wanted to see whether...
For the past week I've been working on Apache Hop, an open-source tool for building data pipelines....

Every Iceberg table accumulates structural debt from its first commit. Files fragment. Snapshots pile...

The AWS Glue Data Catalog's managed table optimizers are the path of least resistance for Iceberg...

Every query against an Iceberg table is a negotiation about how much data the engine is allowed to...