airflow.providers.cncf.kubernetes.utils.pod_manager

Launches PODs

Module Contents

Classes

PodPhase

Possible pod phases

PodLoggingStatus

Used for returning the status of the pod and last log time when exiting from fetch_container_logs

PodManager

Helper class for creating, monitoring, and otherwise interacting with Kubernetes pods

Functions

should_retry_start_pod(exception)

Check if an Exception indicates a transient error and warrants retrying

container_is_running(pod, container_name)

Examines V1Pod pod to determine whether container_name is running.

get_container_termination_message(pod, container_name)

exception airflow.providers.cncf.kubernetes.utils.pod_manager.PodLaunchFailedException[source]

Bases: airflow.exceptions.AirflowException

When pod launching fails in KubernetesPodOperator.

airflow.providers.cncf.kubernetes.utils.pod_manager.should_retry_start_pod(exception)[source]

Check if an Exception indicates a transient error and warrants retrying

class airflow.providers.cncf.kubernetes.utils.pod_manager.PodPhase[source]

Possible pod phases See https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-phase.

PENDING = Pending[source]
RUNNING = Running[source]
FAILED = Failed[source]
SUCCEEDED = Succeeded[source]
terminal_states[source]
airflow.providers.cncf.kubernetes.utils.pod_manager.container_is_running(pod, container_name)[source]

Examines V1Pod pod to determine whether container_name is running. If that container is present and running, returns True. Returns False otherwise.

airflow.providers.cncf.kubernetes.utils.pod_manager.get_container_termination_message(pod, container_name)[source]
class airflow.providers.cncf.kubernetes.utils.pod_manager.PodLoggingStatus[source]

Used for returning the status of the pod and last log time when exiting from fetch_container_logs

running :bool[source]
last_log_time :Optional[pendulum.DateTime][source]
class airflow.providers.cncf.kubernetes.utils.pod_manager.PodManager(kube_client=None, in_cluster=True, cluster_context=None)[source]

Bases: airflow.utils.log.logging_mixin.LoggingMixin

Helper class for creating, monitoring, and otherwise interacting with Kubernetes pods for use with the KubernetesPodOperator

run_pod_async(self, pod, **kwargs)[source]

Runs POD asynchronously

delete_pod(self, pod)[source]

Deletes POD

create_pod(self, pod)[source]

Launches the pod asynchronously.

await_pod_start(self, pod, startup_timeout=120)[source]

Waits for the pod to reach phase other than Pending

Parameters

  • pod (kubernetes.client.models.v1_pod.V1Pod) --

  • startup_timeout (int) -- Timeout (in seconds) for startup of the pod (if pod is pending for too long, fails task)

Returns

Return type

None

follow_container_logs(self, pod, container_name)[source]
fetch_container_logs(self, pod, container_name, *, follow=False, since_time=None)[source]

Follows the logs of container and streams to airflow logging. Returns when container exits.

await_container_completion(self, pod, container_name)[source]
await_pod_completion(self, pod)[source]

Monitors a pod and returns the final state

Parameters

pod (kubernetes.client.models.v1_pod.V1Pod) -- pod spec that will be monitored

Returns

Tuple[State, Optional[str]]

Return type

kubernetes.client.models.v1_pod.V1Pod

parse_log_line(self, line)[source]

Parse K8s log line and returns the final state

Parameters

line (str) -- k8s log line

Returns

timestamp and log message

Return type

Tuple[str, str]

container_is_running(self, pod, container_name)[source]

Reads pod and checks if container is running

read_pod_logs(self, pod, container_name, tail_lines=None, timestamps=False, since_seconds=None, follow=True)[source]

Reads log from the POD

read_pod_events(self, pod)[source]

Reads events from the POD

read_pod(self, pod)[source]

Read POD information

extract_xcom(self, pod)[source]

Retrieves XCom value and kills xcom sidecar container

Was this entry helpful?