Helm Chart for Apache Airflow¶

This chart will bootstrap an Airflow deployment on a Kubernetes cluster using the Helm package manager.

Requirements¶

Kubernetes 1.20+ cluster
Helm 3.0+
PV provisioner support in the underlying infrastructure (optionally)

Features¶

Supported executors: LocalExecutor, CeleryExecutor, CeleryKubernetesExecutor, KubernetesExecutor.
Supported Airflow version: 1.10+, 2.0+
Supported database backend: PostgresSQL, MySQL
Autoscaling for CeleryExecutor provided by KEDA
PostgreSQL and PgBouncer with a battle-tested configuration
Monitoring:
- StatsD/Prometheus metrics for Airflow
- Prometheus metrics for PgBouncer
- Flower
Automatic database migration after a new deployment
Administrator account creation during deployment
Kerberos secure configuration
One-command deployment for any type of executor. You don’t need to provide other services e.g. Redis/Database to test the Airflow.

Installing the Chart¶

To install this chart using Helm 3, run the following commands:

helm repo add apache-airflow https://airflow.apache.org
helm upgrade --install airflow apache-airflow/airflow --namespace airflow --create-namespace

The command deploys Airflow on the Kubernetes cluster in the default configuration. The Parameters reference section lists the parameters that can be configured during installation.

Tip

List all releases using helm list.

Upgrading the Chart¶

To upgrade the chart with the release name airflow:

helm upgrade airflow apache-airflow/airflow --namespace airflow

Note

To upgrade to a new version of the chart, run helm repo update first.

Uninstalling the Chart¶

To uninstall/delete the airflow deployment:

helm delete airflow --namespace airflow

The command removes all the Kubernetes components associated with the chart and deletes the release.

Installing the Chart with ArgoCD¶

When installing the chart using ArgoCD, you MUST set the two following values, or your application will not start as the migrations will not be run:

createUserJob.useHelmHooks: false
migrateDatabaseJob.useHelmHooks: false