airflow.providers.cncf.kubernetes.executors.kubernetes_executor_utils

Module Contents

Classes

ResourceVersion

Singleton for tracking resourceVersion from Kubernetes.

KubernetesJobWatcher

Watches for Kubernetes jobs.

AirflowKubernetesScheduler

Airflow Scheduler for Kubernetes.

Functions

get_base_pod_from_template(pod_template_file, kube_config)

Get base pod from template.
class airflow.providers.cncf.kubernetes.executors.kubernetes_executor_utils.ResourceVersion[source]

Singleton for tracking resourceVersion from Kubernetes.

resource_version: dict[str, str][source]
class airflow.providers.cncf.kubernetes.executors.kubernetes_executor_utils.KubernetesJobWatcher(namespace, watcher_queue, resource_version, scheduler_job_id, kube_config)[source]

Bases: multiprocessing.Process, airflow.utils.log.logging_mixin.LoggingMixin

Watches for Kubernetes jobs.

run()[source]

Perform watching.

process_error(event)[source]

Process error response.

process_status(pod_name, namespace, status, annotations, resource_version, event)[source]

Process status response.

class airflow.providers.cncf.kubernetes.executors.kubernetes_executor_utils.AirflowKubernetesScheduler(kube_config, result_queue, kube_client, scheduler_job_id)[source]

Bases: airflow.utils.log.logging_mixin.LoggingMixin

Airflow Scheduler for Kubernetes.

run_pod_async(pod, **kwargs)[source]

Run POD asynchronously.

run_next(next_job)[source]

Receives the next job to run, builds the pod, and creates it.

delete_pod(pod_name, namespace)[source]

Delete Pod from a namespace; does not raise if it does not exist.

patch_pod_executor_done(*, pod_name, namespace)[source]

Add a “done” annotation to ensure we don’t continually adopt pods.

sync()[source]

Check the status of all currently running kubernetes jobs.

If a job is completed, its status is placed in the result queue to be sent back to the scheduler.

process_watcher_task(task)[source]

Process the task by watcher.

terminate()[source]

Terminates the watcher.

airflow.providers.cncf.kubernetes.executors.kubernetes_executor_utils.get_base_pod_from_template(pod_template_file, kube_config)[source]

Get base pod from template.

Reads either the pod_template_file set in the executor_config or the base pod_template_file set in the airflow.cfg to craft a “base pod” that will be used by the KubernetesExecutor

Parameters

  • pod_template_file (str | None) – absolute path to a pod_template_file.yaml or None

  • kube_config (Any) – The KubeConfig class generated by airflow that contains all kube metadata

Returns

a V1Pod that can be used as the base pod for k8s tasks

Return type

kubernetes.client.models.V1Pod

Was this entry helpful?