Executes task in a Kubernetes POD

Module Contents



Execute a task in a Kubernetes Pod




airflow.providers.cncf.kubernetes.operators.pod.KUBE_CONFIG_ENV_VAR = 'KUBECONFIG'[source]
exception airflow.providers.cncf.kubernetes.operators.pod.PodReattachFailure[source]

Bases: airflow.exceptions.AirflowException

When we expect to be able to find a pod but cannot.

class airflow.providers.cncf.kubernetes.operators.pod.KubernetesPodOperator(*, kubernetes_conn_id=KubernetesHook.default_conn_name, namespace=None, image=None, name=None, random_name_suffix=True, cmds=None, arguments=None, ports=None, volume_mounts=None, volumes=None, env_vars=None, env_from=None, secrets=None, in_cluster=None, cluster_context=None, labels=None, reattach_on_restart=True, startup_timeout_seconds=120, get_logs=True, image_pull_policy=None, annotations=None, container_resources=None, affinity=None, config_file=None, node_selector=None, image_pull_secrets=None, service_account_name=None, is_delete_operator_pod=True, hostnetwork=False, tolerations=None, security_context=None, container_security_context=None, dnspolicy=None, schedulername=None, full_pod_spec=None, init_containers=None, log_events_on_failure=False, do_xcom_push=False, pod_template_file=None, priority_class_name=None, pod_runtime_info_envs=None, termination_grace_period=None, configmaps=None, base_container_name=None, deferrable=False, poll_interval=2, **kwargs)[source]

Bases: airflow.models.BaseOperator

Execute a task in a Kubernetes Pod

See also

For more information on how to use this operator, take a look at the guide: KubernetesPodOperator


If you use Google Kubernetes Engine and Airflow is not running in the same cluster, consider using GKEStartPodOperator, which simplifies the authorization process.

  • kubernetes_conn_id (str | None) – The kubernetes connection id for the Kubernetes cluster.

  • namespace (str | None) – the namespace to run within kubernetes.

  • image (str | None) – Docker image you wish to launch. Defaults to, but fully qualified URLS will point to custom repositories. (templated)

  • name (str | None) – name of the pod in which the task will run, will be used (plus a random suffix if random_name_suffix is True) to generate a pod id (DNS-1123 subdomain, containing only [a-z0-9.-]).

  • random_name_suffix (bool) – if True, will generate a random suffix.

  • cmds (list[str] | None) – entrypoint of the container. (templated) The docker images’s entrypoint is used if this is not provided.

  • arguments (list[str] | None) – arguments of the entrypoint. (templated) The docker image’s CMD is used if this is not provided.

  • ports (list[k8s.V1ContainerPort] | None) – ports for the launched pod.

  • volume_mounts (list[k8s.V1VolumeMount] | None) – volumeMounts for the launched pod.

  • volumes (list[k8s.V1Volume] | None) – volumes for the launched pod. Includes ConfigMaps and PersistentVolumes.

  • env_vars (list[k8s.V1EnvVar] | None) – Environment variables initialized in the container. (templated)

  • env_from (list[k8s.V1EnvFromSource] | None) – (Optional) List of sources to populate environment variables in the container.

  • secrets (list[Secret] | None) – Kubernetes secrets to inject in the container. They can be exposed as environment vars or files in a volume.

  • in_cluster (bool | None) – run kubernetes client with in_cluster configuration.

  • cluster_context (str | None) – context that points to kubernetes cluster. Ignored when in_cluster is True. If None, current-context is used.

  • reattach_on_restart (bool) – if the worker dies while the pod is running, reattach and monitor during the next try. If False, always create a new pod for each try.

  • labels (dict | None) – labels to apply to the Pod. (templated)

  • startup_timeout_seconds (int) – timeout in seconds to startup the pod.

  • get_logs (bool) – get the stdout of the base container as logs of the tasks.

  • image_pull_policy (str | None) – Specify a policy to cache or always pull an image.

  • annotations (dict | None) – non-identifying metadata you can attach to the Pod. Can be a large range of data, and can include characters that are not permitted by labels.

  • container_resources (k8s.V1ResourceRequirements | None) – resources for the launched pod. (templated)

  • affinity (k8s.V1Affinity | None) – affinity scheduling rules for the launched pod.

  • config_file (str | None) – The path to the Kubernetes config file. (templated) If not specified, default value is ~/.kube/config

  • node_selector (dict | None) – A dict containing a group of scheduling rules.

  • image_pull_secrets (list[k8s.V1LocalObjectReference] | None) – Any image pull secrets to be given to the pod. If more than one secret is required, provide a comma separated list: secret_a,secret_b

  • service_account_name (str | None) – Name of the service account

  • is_delete_operator_pod (bool) – What to do when the pod reaches its final state, or the execution is interrupted. If True (default), delete the pod; if False, leave the pod.

  • hostnetwork (bool) – If True enable host networking on the pod.

  • tolerations (list[k8s.V1Toleration] | None) – A list of kubernetes tolerations.

  • security_context (dict | None) – security options the pod should run with (PodSecurityContext).

  • container_security_context (dict | None) – security options the container should run with.

  • dnspolicy (str | None) – dnspolicy for the pod.

  • schedulername (str | None) – Specify a schedulername for the pod

  • full_pod_spec (k8s.V1Pod | None) – The complete podSpec

  • init_containers (list[k8s.V1Container] | None) – init container for the launched Pod

  • log_events_on_failure (bool) – Log the pod’s events if a failure occurs

  • do_xcom_push (bool) – If True, the content of the file /airflow/xcom/return.json in the container will also be pushed to an XCom when the container completes.

  • pod_template_file (str | None) – path to pod template file (templated)

  • priority_class_name (str | None) – priority class name for the launched Pod

  • pod_runtime_info_envs (list[k8s.V1EnvVar] | None) – (Optional) A list of environment variables, to be set in the container.

  • termination_grace_period (int | None) – Termination grace period if task killed in UI, defaults to kubernetes default

  • configmaps (list[str] | None) – (Optional) A list of names of config maps from which it collects ConfigMaps to populate the environment variables with. The contents of the target ConfigMap’s Data field will represent the key-value pairs as environment variables. Extends env_from.

  • base_container_name (str | None) – The name of the base container in the pod. This container’s logs will appear as part of this task’s logs if get_logs is True. Defaults to None. If None, will consult the class variable BASE_CONTAINER_NAME (which defaults to “base”) for the base container name to use.

  • deferrable (bool) – Run operator in the deferrable mode.

  • poll_interval (float) – Polling period in seconds to check for the status. Used only in deferrable mode.

BASE_CONTAINER_NAME = 'base'[source]
POD_CHECKED_KEY = 'already_checked'[source]
template_fields: Sequence[str] = ('image', 'cmds', 'arguments', 'env_vars', 'labels', 'config_file', 'pod_template_file',...[source]
find_pod(namespace, context, *, exclude_checked=True)[source]

Returns an already-running pod for this task instance if one exists.

get_or_create_pod(pod_request_obj, context)[source]

Retrieves xcom value and kills xcom sidecar container


Based on the deferrable parameter runs the pod asynchronously or synchronously


Converts passed config_file to dict format.


Method to easily redefine triggers which are being used in child classes.

execute_complete(context, event, **kwargs)[source]
post_complete_action(*, pod, remote_pod, **kwargs)[source]

Actions that must be done after operator finishes logic of the deferrable_execution.

cleanup(pod, remote_pod)[source]
process_pod_deletion(pod, *, reraise=True)[source]
patch_already_checked(pod, *, reraise=True)[source]

Add an “already checked” annotation to ensure we don’t reattach on retries


Override this method to clean up subprocesses when a task instance gets killed. Any use of the threading, subprocess or multiprocessing module within an operator needs to be cleaned up, or it will leave ghost processes behind.


Returns V1Pod object based on pod template file, full pod spec, and other operator parameters.

The V1Pod attributes are derived (in order of precedence) from operator params, full pod spec, pod template file.


Prints out the pod definition that would be created by this operator. Does not include labels specific to the task instance (since there isn’t one in a dry_run) and excludes all empty elements.

Was this entry helpful?