Parameters reference¶
The following tables lists the configurable parameters of the Airflow chart and their default values.
Sections:
Common¶
Parameter |
Description |
Default |
---|---|---|
|
airflow_local_settings file as a string. |
|
|
Airflow version (Used to make some decisions based on Airflow Version being deployed). |
|
|
Settings to go into the mounted airflow.cfg |
|
|
Default airflow repository. Overrides all the specific images below. |
|
|
Default airflow tag to deploy. |
|
|
Airflow executor. |
|
|
The Fernet key used to encrypt passwords. |
|
|
Enable PgBouncer. |
|
|
Enable default user creation. |
|
|
This will be mounted into the Airflow webserver as a custom webserver_config.py. You can bake a webserver_config.py in to your image instead. |
Examples: |
Airflow¶
Parameter |
Description |
Default |
---|---|---|
|
Airflow home directory. Used for mount paths. |
|
|
Whether various Airflow components launch pods. |
|
|
Git branch |
|
|
Git sync container name. |
|
|
Name of a Secret containing the repo GIT_SYNC_USERNAME and GIT_SYNC_PASSWORD. |
|
|
Repository depth. |
|
|
Destination folder. |
|
|
Enable Git sync. |
|
|
Environment variables for git sync container. |
Examples: |
|
Disable Git sync on webserver as it is not needed when DAG Serialization is enabled. |
|
|
Mount additional volumes into git sync container. |
|
|
When using a ssh private key, the contents of your known_hosts file. |
Examples: |
|
The number of consecutive failures allowed before aborting. |
|
|
Git repository. |
|
|
Git revision. |
|
|
Root directory. |
|
|
Name of a Secret containing the repo sshKeySecret. |
|
|
Subpath within the repo where dags are located. |
|
|
Git sync container run as user parameter. |
|
|
Interval between git sync attempts in seconds. |
|
|
Access mode of the persistent volume. |
|
|
Enable persistent volume for storing dags. |
|
|
The name of an existing PVC to use. |
|
|
Volume size for dags. |
|
|
If using a custom StorageClass, pass name here. |
|
|
Elasticsearch connection configuration. |
Examples: |
|
Enable Elasticsearch task logging. |
|
|
A secret containing the connection string. |
|
|
Environment variables for all Airflow containers. |
Examples: |
|
Extra env ‘items’ that will be added to the definition of Airflow containers; a string is expected (can be templated). |
Examples: |
|
Extra envFrom ‘items’ that will be added to the definition of Airflow containers; a string is expected (can be templated). |
Examples: |
|
The Fernet key secret name. |
|
|
Group of airflow user. |
|
|
Enable persistent volume for storing logs. |
|
|
The name of an existing PVC to use. |
|
|
Volume size for logs. |
|
|
If using a custom StorageClass, pass name here. |
|
|
Whether the KubernetesExecutor can launch workers and pods in multiple namespaces |
|
|
pod_template used for KubernetesExecutor workers |
|
|
Secrets for all Airflow containers. |
Examples: |
|
User of airflow user. |
|
Images¶
Parameter |
Description |
Default |
---|---|---|
|
The airflow image pull policy. |
|
|
The airflow image repository. |
|
|
The airflow image tag. |
|
|
The flower image pull policy. |
|
|
The flower image repository. |
|
|
The flower image tag. |
|
|
The gitSync image pull policy. |
|
|
The gitSync image repository. |
|
|
The gitSync image tag. |
|
|
The PgBouncer image pull policy. |
|
|
The PgBouncer image repository. |
|
|
The PgBouncer image tag. |
|
|
The PgBouncer exporter image pull policy. |
|
|
The PgBouncer exporter image repository. |
|
|
The PgBouncer exporter image tag. |
|
|
The pod_template image pull policy. |
|
|
The pod_template image repository. |
|
|
The pod_template image tag. |
|
|
The redis image pull policy. |
|
|
The redis image repository. |
|
|
The redis image tag. |
|
|
The statsd image pull policy. |
|
|
The statsd image repository. |
|
|
The statsd image tag. |
|
Ports¶
Parameter |
Description |
Default |
---|---|---|
|
Airflow UI port. |
|
|
Flower UI port. |
|
|
PgBouncer port. |
|
|
PgBouncer scrape port. |
|
|
Redis port. |
|
|
StatsD ingest port. |
|
|
StatsD scrape port. |
|
|
Worker logs port. |
|
Database¶
Parameter |
Description |
Default |
---|---|---|
|
The name of the database. |
|
|
The database host. |
|
|
The user’s password. |
|
|
The database port. |
|
|
The database protocol. |
|
|
The database SSL parameter. |
|
|
The database user. |
|
|
Metadata connection string secret. |
|
|
Result backend connection configuration. |
|
|
The name of the database. |
|
|
The database host. |
|
|
The database password. |
|
|
The database port. |
|
|
The database protocol. |
|
|
The database SSL parameter. |
|
|
The database user. |
|
|
Result backend connection string secret. |
|
|
Enable PostgreSQL subchart. |
|
|
PostgreSQL password. |
|
|
PostgreSQL username. |
|
PgBouncer¶
Parameter |
Description |
Default |
---|---|---|
|
Specify scheduling constraints for PgBouncer pods. |
|
|
The allowed ciphers, might be ‘fast’, ‘normal’ or list ciphers separated with ‘:’. |
|
|
The PgBouncer config Secret name. |
|
|
Additional NetworkPolicies as needed. |
|
|
Log successful logins. |
|
|
Log disconnections with reasons. |
|
|
Maximum clients that can connect to PgBouncer (higher = more file descriptors). |
|
|
Metadata pool size. |
|
|
Select certain nodes for PgBouncer pods. |
|
|
Max unavailable pods for PgBouncer. |
|
|
Enabled PodDistributionBudget. |
|
|
Resources for the PgBouncer pods. |
Examples: |
|
Result backend pool size. |
|
|
Extra annotations for the PgBouncer Service. |
|
|
Annotations to add to the worker Kubernetes ServiceAccount. |
|
|
Specifies whether a ServiceAccount should be created. |
|
|
The name of the ServiceAccount to use. If not set and create is true, a name is generated using the release name. |
|
|
Certificate Authority for server side |
|
|
Server Certificate for server side |
|
|
Private key used to authenticate with the server |
|
|
SSL mode for PgBouncer. |
|
|
Specify Tolerations for PgBouncer pods. |
|
|
PgBouncer run as user parameter. |
|
|
Increase PgBouncer verbosity. |
|
Scheduler¶
Parameter |
Description |
Default |
---|---|---|
|
Specify scheduling constraints for scheduler pods. |
|
|
Launch additional containers into scheduler. |
|
|
Mount additional volumes into scheduler. |
|
|
Mount additional volumes into scheduler. |
|
|
Scheduler Liveness probe failure threshold. |
|
|
Scheduler Liveness probe initial delay. |
|
|
Webserver Liveness probe period seconds. |
|
|
Scheduler Liveness probe timeout seconds. |
|
|
Select certain nodes for scheduler pods. |
|
|
Max unavailable pods for scheduler. |
|
|
Enable pod disruption budget. |
|
|
Airflow 2.0 allows users to run multiple schedulers. This feature is only recommended for MySQL 8+ and PostgreSQL |
|
|
Resources for scheduler pods. |
Examples: |
|
This setting tells Kubernetes that its ok to evict when it wants to scale a node down. |
|
|
Annotations to add to the scheduler Kubernetes ServiceAccount. |
|
|
Specifies whether a ServiceAccount should be created. |
|
|
The name of the ServiceAccount to use. If not set and create is true, a name is generated using the release name. |
|
|
Specify Tolerations for scheduler pods. |
|
Webserver¶
Parameter |
Description |
Default |
---|---|---|
|
Specify scheduling constraints for webserver pods. |
|
|
Allow webserver to read k8s pod logs. Useful when you don’t have an external log store. |
|
|
Default user email address. |
|
|
Default user firstname. |
|
|
Default user lastname. |
|
|
Default user password. |
|
|
Default user role. |
|
|
Default user username. |
|
|
Launch additional containers into webserver. |
|
|
Additional NetworkPolicies as needed. |
|
|
Mount additional volumes into webserver. |
|
|
Mount additional volumes into webserver. |
|
|
Webserver Liveness probe failure threshold. |
|
|
Webserver Liveness probe initial delay. |
|
|
Webserver Liveness probe period seconds. |
|
|
Webserver Liveness probe timeout seconds. |
|
|
Select certain nodes for webserver pods. |
|
|
Webserver Readiness probe failure threshold. |
|
|
Webserver Readiness probe initial delay. |
|
|
Webserver Readiness probe period seconds. |
|
|
Webserver Readiness probe timeout seconds. |
|
|
How many Airflow webserver replicas should run. |
|
|
Resources for webserver pods. |
Examples: |
|
Annotations for the webserver Service. |
|
|
Webserver Service type. |
|
|
Annotations to add to the webserver Kubernetes ServiceAccount. |
|
|
Specifies whether a ServiceAccount should be created. |
|
|
The name of the ServiceAccount to use. If not set and create is true, a name is generated using the release name. |
|
|
Specifies the strategy used to replace old Pods by new ones. |
|
|
Specify Tolerations for webserver pods. |
|
Workers¶
Parameter |
Description |
Default |
---|---|---|
|
Specify scheduling constraints for worker pods. |
|
|
Launch additional containers into workers. |
|
|
Mount additional volumes into workers. |
|
|
Mount additional volumes into workers. |
|
|
Specify HostAliases for workers. |
Examples: |
|
How many seconds KEDA will wait before scaling to zero. |
|
|
Allow KEDA autoscaling. Persistence.enabled must be set to false to use KEDA. |
|
|
Maximum number of workers created by KEDA. |
|
|
Labels used in matchLabels for namespace in the PgBouncer NetworkPolicy. |
|
|
How often KEDA polls the airflow DB to report new scale requests to the HPA. |
|
|
Enable Kerberos sidecar for the worker. |
|
|
Resources on workers kerberos sidecar |
Examples: |
|
Select certain nodes for worker pods. |
|
|
Enable persistent volumes. |
|
|
Execute init container to chown log directory. This is currently only needed in kind, due to usage of local-path provisioner. |
|
|
Volume size for worker StatefulSet. |
|
|
If using a custom StorageClass, pass name ref to all StatefulSets here. |
|
|
Number of Airflow Celery workers in StatefulSet. |
|
|
Resources on workers |
Examples: |
|
This setting tells Kubernetes that it’s ok to evict when it wants to scale a node down. |
|
|
Annotations to add to the worker Kubernetes ServiceAccount. |
|
|
Specifies whether a ServiceAccount should be created. |
|
|
The name of the ServiceAccount to use. If not set and create is true, a name is generated using the release name. |
|
|
Specifies the strategy used to replace old Pods by new ones when deployed as a Deployment. |
|
|
Grace period for tasks to finish after SIGTERM is sent from Kubernetes. |
|
|
Specify Tolerations for worker pods. |
|
|
Specifies the strategy used to replace old Pods by new ones when deployed as a StatefulSet. |
|
Flower¶
Parameter |
Description |
Default |
---|---|---|
|
Specify scheduling constraints for Flower pods. |
|
|
Enable Flower. |
|
|
Additional NetworkPolicies as needed. |
|
|
Select certain nodes for Flower pods. |
|
|
Password use to access Flower. |
|
|
Resources for Flower pods. |
Examples: |
|
A secret containing the user and password pair. |
|
|
Flower Service type. |
|
|
Annotations to add to the worker Kubernetes ServiceAccount. |
|
|
Specifies whether a ServiceAccount should be created. |
|
|
The name of the ServiceAccount to use. If not set and create is true, a name is generated using the release name. |
|
|
Specify Tolerations for Flower pods. |
|
|
Username use to access Flower. |
|
Redis¶
Parameter |
Description |
Default |
---|---|---|
|
Direct url to the redis broker (when using an external redis instance). |
|
|
Redis broker URL secret. |
|
|
Specify scheduling constraints for Redis pods. |
|
|
Enable the Redis provisioned by the chart (you can also use an external Redis instance with data.brokerUrl or data.brokerUrlSecretName). |
|
|
Select certain nodes for Redis pods. |
|
|
If password is set, create secret with it, else generate a new one on install. |
|
|
Redis password secret. |
|
|
Enable persistent volumes. |
|
|
Volume size for Redis StatefulSet. |
|
|
If using a custom StorageClass, pass name ref to all StatefulSets here. |
|
|
Resources for the Redis pods |
Examples: |
|
This setting tells Kubernetes that its ok to evict when it wants to scale a node down. |
|
|
Annotations to add to the worker Kubernetes ServiceAccount. |
|
|
Specifies whether a ServiceAccount should be created. |
|
|
The name of the ServiceAccount to use. If not set and create is true, a name is generated using the release name. |
|
|
Grace period for Redis to exit after SIGTERM is sent from Kubernetes. |
|
|
Specify Tolerations for Redis pods. |
|
Statsd¶
Parameter |
Description |
Default |
---|---|---|
|
Specify scheduling constraints for StatsD pods. |
|
|
Enable StatsD. |
|
|
Additional mappings for StatsD exporter. |
|
|
Additional NetworkPolicies as needed. |
|
|
Select certain nodes for StatsD pods. |
|
|
Resources for StatsD pods. |
Examples: |
|
Extra annotations for the StatsD Service. |
|
|
Annotations to add to the StatsD Kubernetes ServiceAccount. |
|
|
Specifies whether a ServiceAccount should be created. |
|
|
The name of the ServiceAccount to use. If not set and create is true, a name is generated using the release name. |
|
|
Specify Tolerations for StatsD pods. |
|
|
StatsD run as user parameter. |
|
Jobs¶
Parameter |
Description |
Default |
---|---|---|
|
Specify scheduling constraints for cleanup pods. |
|
|
Enable cleanup. |
|
|
Select certain nodes for cleanup pods. |
|
|
Cleanup schedule. |
|
|
Annotations to add to the cleanup CronJob Kubernetes ServiceAccount. |
|
|
Specifies whether a ServiceAccount should be created. |
|
|
The name of the ServiceAccount to use. If not set and create is true, a name is generated using the release name. |
|
|
Specify Tolerations for cleanup pods. |
|
|
Annotations to add to the create user job Kubernetes ServiceAccount. |
|
|
Specifies whether a ServiceAccount should be created. |
|
|
The name of the ServiceAccount to use. If not set and create is true, a name is generated using the release name. |
|
|
Annotations to add to the migrate database job Kubernetes ServiceAccount. |
|
|
Specifies whether a ServiceAccount should be created. |
|
|
The name of the ServiceAccount to use. If not set and create is true, a name is generated using the release name. |
|
Kubernetes¶
Parameter |
Description |
Default |
---|---|---|
|
Specify scheduling constraints for all pods. |
|
|
Extra annotations to apply to the main Airflow configmap. |
|
|
Extra annotations to apply to all Airflow pods. |
|
|
Extra ConfigMaps that will be managed by the chart. |
Examples: |
|
Extra secrets that will be managed by the chart. |
Examples: |
|
Add common labels to all objects and pods defined in this chart. |
|
|
Define default/max/min values for pods and containers in namespace. |
|
|
Enabled network policies. |
|
|
Select certain nodes for all pods. |
|
|
Define any ResourceQuotas for namespace. |
|
|
Specifies whether RBAC resources should be created. |
|
|
Registry connection configuration. |
Examples: |
|
Registry connection string secret. |
|
|
Specify Tolerations for all pods. |
|
Ingress¶
Parameter |
Description |
Default |
---|---|---|
|
Enable ingress resource. |
|
|
Annotations for the flower Ingress. |
|
|
The hostname for the flower Ingress. |
|
|
The path for the flower Ingress. |
|
|
HTTP paths to add to the flower Ingress before the default path. |
|
|
HTTP paths to add to the flower Ingress after the default path. |
|
|
Enable TLS termination for the flower Ingress. |
|
|
The name of a pre-created Secret containing a TLS private key and certificate. |
|
|
Annotations for the web Ingress. |
|
|
The hostname for the web Ingress. |
|
|
The path for the web Ingress. |
|
|
HTTP paths to add to the web Ingress before the default path. |
|
|
HTTP paths to add to the web Ingress after the default path. |
|
|
Enable TLS termination for the web Ingress. |
|
|
The name of a pre-created Secret containing a TLS private key and certificate. |
|
Kerberos¶
Parameter |
Description |
Default |
---|---|---|
|
Name for kerberos credentials cache file. |
|
|
Path to mount shared volume for kerberos credentials cache. |
|
|
Contents of krb5.conf. |
|
|
Path to mount krb5.conf kerberos configuration file. |
|
|
Enable kerberos. |
|
|
Path to mount the keytab for refreshing credentials in the kerberos sidecar. |
|
|
Principal to use when refreshing kerberos credentials. |
|
|
How often (in seconds) airflow kerberos will reinitialize the credentials cache. |
|
Specify each parameter using the --set key=value[,key=value]
argument to helm install
. For example,