
How I Built a Two-Tier Data Platform in 9 Phases (Spark, Kafka, dbt, Kubernetes)
A complete data platform — batch + streaming, single-node + distributed, local + Kubernetes. What I built, what broke, and what I learned.
Tag archive

A complete data platform — batch + streaming, single-node + distributed, local + Kubernetes. What I built, what broke, and what I learned.
In distributed microservice architectures, synchronizing database updates with downstream event...

Welcome to another article in Batch Processing. We have covered Object Stores and Distributed File...

A post by Shitanshu Jha

Understand Kafka's partition assignment strategies, how slow consumers create Head-of-Line blocking, and how to implement parallel decoupled worker architectures.

Articles 🔍 API Features as a First-Class Artifact with OpenAPI Overlays OpenAPI Overlay...
Working with Kafka data on your own terms, using a local index. A search stack usually follows Kafka...
What actually breaks without a transactional outbox, why a ledger nobody reads back isn't really double-entry, and what happens once you decide never to auto-correct money.

TL;DR: One bad message can stop a Kafka consumer, because offsets are committed in order: if record...
Event-Driven Architecture: Building Responsive, Scalable Systems ...
Primeiro post de uma série sobre arquitetura orientada a eventos (EDA), construída em público a...
Introduction Duplicate processing remains a leading source of production incidents....