Implements Docker operator

Module Contents

class airflow.providers.docker.operators.docker.DockerOperator(*, image: str, api_version: Optional[str] = None, command: Optional[Union[str, List[str]]] = None, container_name: Optional[str] = None, cpus: float = 1.0, docker_url: str = 'unix://var/run/docker.sock', environment: Optional[Dict] = None, private_environment: Optional[Dict] = None, force_pull: bool = False, mem_limit: Optional[Union[float, str]] = None, host_tmp_dir: Optional[str] = None, network_mode: Optional[str] = None, tls_ca_cert: Optional[str] = None, tls_client_cert: Optional[str] = None, tls_client_key: Optional[str] = None, tls_hostname: Optional[Union[str, bool]] = None, tls_ssl_version: Optional[str] = None, tmp_dir: str = '/tmp/airflow', user: Optional[Union[str, int]] = None, volumes: Optional[List[str]] = None, entrypoint: Optional[Union[str, List[str]]] = None, working_dir: Optional[str] = None, xcom_all: bool = False, docker_conn_id: Optional[str] = None, dns: Optional[List[str]] = None, dns_search: Optional[List[str]] = None, auto_remove: bool = False, shm_size: Optional[int] = None, tty: bool = False, privileged: bool = False, cap_add: Optional[Iterable[str]] = None, extra_hosts: Optional[Dict[str, str]] = None, **kwargs)[source]

Bases: airflow.models.BaseOperator

Execute a command inside a docker container.

A temporary directory is created on the host and mounted into a container to allow storing files that together exceed the default disk size of 10GB in a container. The path to the mounted directory can be accessed via the environment variable AIRFLOW_TMP_DIR.

If a login to a private registry is required prior to pulling the image, a Docker connection needs to be configured in Airflow and the connection ID be provided with the parameter docker_conn_id.

  • image (str) -- Docker image from which to create the container. If image tag is omitted, "latest" will be used.

  • api_version (str) -- Remote API version. Set to auto to automatically detect the server's version.

  • command (str or list) -- Command to be run in the container. (templated)

  • container_name (str or None) -- Name of the container. Optional (templated)

  • cpus (float) -- Number of CPUs to assign to the container. This value gets multiplied with 1024. See

  • docker_url (str) -- URL of the host running the docker daemon. Default is unix://var/run/docker.sock

  • environment (dict) -- Environment variables to set in the container. (templated)

  • private_environment (dict) -- Private environment variables to set in the container. These are not templated, and hidden from the website.

  • force_pull (bool) -- Pull the docker image on every run. Default is False.

  • mem_limit (float or str) -- Maximum amount of memory the container can use. Either a float value, which represents the limit in bytes, or a string like 128m or 1g.

  • host_tmp_dir (str) -- Specify the location of the temporary directory on the host which will be mapped to tmp_dir. If not provided defaults to using the standard system temp directory.

  • network_mode (str) -- Network mode for the container.

  • tls_ca_cert (str) -- Path to a PEM-encoded certificate authority to secure the docker connection.

  • tls_client_cert (str) -- Path to the PEM-encoded certificate used to authenticate docker client.

  • tls_client_key (str) -- Path to the PEM-encoded key used to authenticate docker client.

  • tls_hostname (str or bool) -- Hostname to match against the docker server certificate or False to disable the check.

  • tls_ssl_version (str) -- Version of SSL to use when communicating with docker daemon.

  • tmp_dir (str) -- Mount point inside the container to a temporary directory created on the host by the operator. The path is also made available via the environment variable AIRFLOW_TMP_DIR inside the container.

  • user (int or str) -- Default user inside the docker container.

  • volumes (list) -- List of volumes to mount into the container, e.g. ['/host/path:/container/path', '/host/path2:/container/path2:ro'].

  • entrypoint (str or list) -- Overwrite the default ENTRYPOINT of the image

  • working_dir (str) -- Working directory to set on the container (equivalent to the -w switch the docker client)

  • xcom_all (bool) -- Push all the stdout or just the last line. The default is False (last line).

  • docker_conn_id (str) -- The Docker connection id

  • dns (list[str]) -- Docker custom DNS servers

  • dns_search (list[str]) -- Docker custom DNS search domain

  • auto_remove (bool) -- Auto-removal of the container on daemon side when the container's process exits. The default is False.

  • shm_size (int) -- Size of /dev/shm in bytes. The size must be greater than 0. If omitted uses system default.

  • tty (bool) -- Allocate pseudo-TTY to the container This needs to be set see logs of the Docker container.

  • privileged (bool) -- Give extended privileges to this container.

  • cap_add (list[str]) -- Include container capabilities

template_fields = ['command', 'environment', 'container_name'][source]
template_ext = ['.sh', '.bash'][source]

Retrieves hook for the operator.


The Docker Hook


Run a Docker container with the provided image

execute(self, context)[source]
static format_command(command: Union[str, List[str]])[source]

Retrieve command(s). if command string starts with [, it returns the command list)


command (str | List[str]) -- Docker command or entrypoint


the command (or commands)

Return type

str | List[str]


Was this entry helpful?