airflow.contrib.operators.kubernetes_pod_operator

Module Contents

class airflow.contrib.operators.kubernetes_pod_operator.KubernetesPodOperator(namespace, image, name, cmds=None, arguments=None, ports=None, volume_mounts=None, volumes=None, env_vars=None, secrets=None, in_cluster=False, cluster_context=None, labels=None, startup_timeout_seconds=120, get_logs=True, image_pull_policy='IfNotPresent', annotations=None, resources=None, affinity=None, config_file=None, xcom_push=False, node_selectors=None, image_pull_secrets=None, service_account_name='default', is_delete_operator_pod=False, hostnetwork=False, tolerations=None, configmaps=None, security_context=None, pod_runtime_info_envs=None, dnspolicy=None, *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Execute a task in a Kubernetes Pod

Parameters

  • image (str) – Docker image you wish to launch. Defaults to dockerhub.io, but fully qualified URLS will point to custom repositories

  • namespace (str) – the namespace to run within kubernetes

  • cmds (list[str]) – entrypoint of the container. (templated) The docker images’s entrypoint is used if this is not provide.

  • arguments (list[str]) – arguments of the entrypoint. (templated) The docker image’s CMD is used if this is not provided.

  • image_pull_policy (str) – Specify a policy to cache or always pull an image

  • image_pull_secrets (str) – Any image pull secrets to be given to the pod. If more than one secret is required, provide a comma separated list: secret_a,secret_b

  • ports (list[airflow.contrib.kubernetes.pod.Port]) – ports for launched pod

  • volume_mounts (list[airflow.contrib.kubernetes.volume_mount.VolumeMount]) – volumeMounts for launched pod

  • volumes (list[airflow.contrib.kubernetes.volume.Volume]) – volumes for launched pod. Includes ConfigMaps and PersistentVolumes

  • labels (dict) – labels to apply to the Pod

  • startup_timeout_seconds (int) – timeout in seconds to startup the pod

  • name (str) – name of the task you want to run, will be used to generate a pod id

  • env_vars (dict) – Environment variables initialized in the container. (templated)

  • secrets (list[airflow.contrib.kubernetes.secret.Secret]) – Kubernetes secrets to inject in the container, They can be exposed as environment vars or files in a volume.

  • in_cluster (bool) – run kubernetes client with in_cluster configuration

  • cluster_context (str) – context that points to kubernetes cluster. Ignored when in_cluster is True. If None, current-context is used.

  • get_logs (bool) – get the stdout of the container as logs of the tasks

  • annotations (dict) – non-identifying metadata you can attach to the Pod. Can be a large range of data, and can include characters that are not permitted by labels.

  • resources (dict) – A dict containing a group of resources requests and limits

  • affinity (dict) – A dict containing a group of affinity scheduling rules

  • node_selectors (dict) – A dict containing a group of scheduling rules

  • config_file (str) – The path to the Kubernetes config file

  • xcom_push (bool) – If xcom_push is True, the content of the file /airflow/xcom/return.json in the container will also be pushed to an XCom when the container completes.

  • is_delete_operator_pod (bool) – What to do when the pod reaches its final state, or the execution is interrupted. If False (default): do nothing, If True: delete the pod

  • hostnetwork (bool) – If True enable host networking on the pod

  • tolerations (list tolerations) – A list of kubernetes tolerations

  • configmaps (list[str]) – A list of configmap names objects that we want mount as env variables

  • pod_runtime_info_envs (list[PodRuntimeEnv]) – environment variables about pod runtime information (ip, namespace, nodeName, podName)

  • dnspolicy (str) – Specify a dnspolicy for the pod

template_fields = ['cmds', 'arguments', 'env_vars', 'config_file'][source]
execute(self, context)[source]
_set_resources(self, resources)[source]