
Time in Apache Flink: watermarks, idle partitions, state TTL and replay
Silently dropped late events, a quiet partition stalling every window, TTL on the wall clock, and processing-time timers that break deterministic replay.
Tag archive

Silently dropped late events, a quiet partition stalling every window, TTL on the wall clock, and processing-time timers that break deterministic replay.

Where an event pipeline loses or double-counts data: idempotency keys claimed too early, sink buffers outside checkpoints, and materialized views.

Building real-time data streaming architectures requires sub-second processing latencies, robust...

A single null-key record crashed a whole Stateful Functions job. KZM-3.5 ships invalidRecordHandling - skip by default, strict fail per topic, full per-record diagnostics.
Apache Stateful Functions never let a function see Kafka record headers, and never let it set them....
Martin Tuncaydin explores building real-time travel data platforms using Apache Kafka and Flink to solve inventory sync, pricing, and overbooking challeng…

Apache Stateful Functions hasn't shipped a release since October 2024 and pins Flink 1.16 + Java 11. We needed it on Flink 2.x with Java 21, plus Kinesis I/O restored. Here's the rebuild.

Flink added support for what it calls Materialized Tables in 1.20, released in 2024. You can read...

Author: xiaojing Last Updated: 2026-03-11 Overview In this tutorial, you will learn how to...
Martin Tuncaydin explores building real-time travel data platforms using Apache Kafka and Flink to capture booking events, inventory changes, and pricing …
Week 7 of Data Engineering Zoomcamp by @DataTalksClub complete Just finished Module 7 - Streaming...

Introduction When you are working with Kafka, there are plenty of ways out there to move...