Apache Airflow
the six Ws · specification
The Apache Software Foundation and its open-source community maintain Apache Airflow.
A platform to programmatically author, schedule, and monitor workflows defined as Python DAGs.
Self-hosted on-premises or Kubernetes, or consumed via managed offerings like Astronomer, AWS MWAA, and Google Cloud Composer.
Originally created at Airbnb in 2014, became an Apache top-level project in 2019, and remains actively developed through 2026.
Provides mature, code-first orchestration for ETL, data pipelines, and ML workflows with dependency management and a web UI.
Written in Python, integrates with a wide ecosystem of provider packages for cloud services, databases, and messaging systems.
Open-source (Apache 2.0 license), self-hosted by default, data and credentials remain on the deployer's own infrastructure unless a managed vendor is used.