
Apache Iceberg v4 Column Updates Are About to Change Feature Engineering
Iceberg v4 is adding column files, companion files that can hold one or more whole columns for every row of a data file and be swapped in by...
Tag archive

Iceberg v4 is adding column files, companion files that can hold one or more whole columns for every row of a data file and be swapped in by...
Introduction In the world of data processing, Parquet has emerged as a cornerstone format...
For a decade the rule was simple: small data goes in pandas, big data goes in a warehouse, and...
Ein Jahrzehnt lang galt eine einfache Regel: Kleine Daten kommen in pandas, große in ein Data...
Imagine asking an AI assistant for the average order value in a 12 GB sales export. It finds an...
You get a .parquet file and need to check what's in it. No Python, no Spark, ideally without...

The registry behind Iceberg's File Format API is keyed on a closed enum of formats and an open set of engine row types. This blog explains the API...

A comprehensive breakdown of Apache Parquet's internal architecture, explaining why it became the universal standard for modern data engineering and how to leverage cardinality sorting to dramatically reduce storage costs.

How Coperni serves the Copernicus air quality forecast for all of Europe with one daily Parquet file, DuckDB and HTTP range requests, for well under a euro a month.

Aug 25, 2026 · engineering Hybrid search runs two retrievers over one query. A keyword retriever...

TL;DR Working in AWS S3 / Athena, I constantly wanted to quickly inspect a Parquet file...
How I run 90-day video analytics over Parquet exports with DuckDB — no warehouse, no server. Covers