#airflow

If you've recently upgraded to Apache Airflow 3 and logged into the webserver, you may have noticed...

Build a Data Pipeline with Apache Airflow Build a Data Pipeline with Apache...

Roughly 70% of production data pipelines in the healthcare and fintech sectors I’ve audited are...

Level: 300–400 | Reading time: ~10 minutes Recently, a customer asked me when they should choose...

Building Production-Ready Data Pipelines with Python Data pipelines that work in...

Introduction: I Built a Pipeline. Now What? In the previous article Understanding ETL: A...

Develop and test Apache Airflow DAGs locally with Leoflow Lite — no Docker, no Kubernetes, no heavy stack. Real Airflow 3.2 operators, sensors, connections and variables, hot-reload on save, the Airflow UI, and your local GCP/AWS/Azure credentials just work.

Leoflow's control plane is Go and never imports Apache Airflow — yet it reads standard airflow.sdk DAGs. The trick is a dependency-free structural shim that exec's your dag.py, records the graph, and lets the real provider operator run later in the pod. Here's the whole mechanism, end to end.

Intro Last year, I joined a team responsible for a mature data platform with several tools...

Write standard Apache Airflow 3.2 DAGs in Python; Leoflow parses them with a dependency-free shim, runs each task as its own pod, and serves the Airflow UI — with a Go control plane that has no GIL and no Airflow in the scheduling path. v0.1.0 ships the shim, 86 connectors, generic provider operators + sensors, and a resilient local Lite edition.

What Is Airflow? Let me be honest about how I first encountered Airflow. My team had a...

content/posts/airflow-pipelines.md