Tag archive
Airflow vs Dagster vs Prefect: compare tasks, assets and flows, scheduling, lineage, testing, deployment, managed options and migration, with one ETL in each.
This article was first published on the Bruin blog. A data orchestrator is the system that runs a...
Notes from the second course: source systems, batch vs streaming ingestion, ETL vs ELT, and orchestrating pipelines with Airflow on AWS.
A ZoomEye fingerprint query for Apache Airflow returned 73,265 matching hosts. The article covers why a workflow platform concentrates credentials and which defaults matter.
If you upgrade a DAG like this from Airflow 2 to Airflow 3 without changing a line, it can start...
A green check means it ran, not that the data is right. How to make correctness its own, loud signal.
How we collect health metadata from Airflow, Glue, dbt and Tableau into S3, serve it through a thin...
Three delivery toolchain products return fingerprints at different scales, and exposure matters most for the one that serves the artefacts other systems install.
Disclosure: I wrote this handbook. It is free and MIT licensed. Tutorials rot quietly. A dependency...
We deployed Windmill with Docker Compose, built a mixed-language data and AI workflow, tested worker isolation, and mapped the operational trade-offs against Airflow.
Practical Airflow DAG design patterns for production: idempotency, atomic tasks, dynamic mapping, backfills, sensors, secrets, and testing DAGs.
We started using Apache Airflow for repetitive DevOps workflows, not just data pipelines. Some of...