Using additional containers¶

Sidecar Containers¶

If you want to deploy your own sidecar container, you can add it through the extraContainers parameter. You can define different containers for the scheduler, webserver, worker, triggerer, DAG processor, flower, create user Job and migrate database Job Pods.

For example, sidecars that sync DAGs from object storage.

scheduler:
  extraContainers:
    - name: s3-sync
      image: my-company/s3-sync:latest
      imagePullPolicy: Always
workers:
  extraContainers:
    - name: s3-sync
      image: my-company/s3-sync:latest
      imagePullPolicy: Always

Note

If you use workers.extraContainers with KubernetesExecutor, you are responsible for signaling sidecars to exit when the main container finishes so Airflow can continue the worker shutdown process!

Init Containers¶

You can also deploy extra init containers through the extraInitContainers parameter. You can define different containers for the scheduler, webserver, worker, triggerer and DAG processor pods.

For example, an init container that just says hello:

scheduler:
  extraInitContainers:
    - name: hello
      image: debian
      args:
        - echo
        - hello