`airflow.providers.cncf.kubernetes.utils.pod_manager`¶

Launches PODs

Module Contents¶

Classes¶

`PodPhase`	Possible pod phases
`PodManager`	Helper class for creating, monitoring, and otherwise interacting with Kubernetes pods

Functions¶

`should_retry_start_pod`(exception: Exception) → bool	Check if an Exception indicates a transient error and warrants retrying
`container_is_running`(pod: kubernetes.client.models.v1_pod.V1Pod, container_name: str) → bool	Examines V1Pod `pod` to determine whether `container_name` is running.

exception airflow.providers.cncf.kubernetes.utils.pod_manager.PodLaunchFailedException[source]¶

Bases: airflow.exceptions.AirflowException

When pod launching fails in KubernetesPodOperator.

airflow.providers.cncf.kubernetes.utils.pod_manager.should_retry_start_pod(exception: Exception) → bool[source]¶: Check if an Exception indicates a transient error and warrants retrying

class airflow.providers.cncf.kubernetes.utils.pod_manager.PodPhase[source]¶

Possible pod phases See https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-phase.

PENDING = Pending[source]¶

RUNNING = Running[source]¶

FAILED = Failed[source]¶

SUCCEEDED = Succeeded[source]¶

terminal_states[source]¶

airflow.providers.cncf.kubernetes.utils.pod_manager.container_is_running(pod: kubernetes.client.models.v1_pod.V1Pod, container_name: str) → bool[source]¶: Examines V1Pod pod to determine whether container_name is running. If that container is present and running, returns True. Returns False otherwise.

class airflow.providers.cncf.kubernetes.utils.pod_manager.PodManager(kube_client: kubernetes.client.CoreV1Api = None, in_cluster: bool = True, cluster_context: Optional[str] = None)[source]¶

Bases: airflow.utils.log.logging_mixin.LoggingMixin

Helper class for creating, monitoring, and otherwise interacting with Kubernetes pods for use with the KubernetesPodOperator

run_pod_async(self, pod: kubernetes.client.models.v1_pod.V1Pod, **kwargs) → kubernetes.client.models.v1_pod.V1Pod[source]¶: Runs POD asynchronously

delete_pod(self, pod: kubernetes.client.models.v1_pod.V1Pod) → None[source]¶: Deletes POD

create_pod(self, pod: kubernetes.client.models.v1_pod.V1Pod) → kubernetes.client.models.v1_pod.V1Pod[source]¶: Launches the pod asynchronously.

await_pod_start(self, pod: kubernetes.client.models.v1_pod.V1Pod, startup_timeout: int = 120) → None[source]¶

Waits for the pod to reach phase other than Pending

Parameters

pod --
startup_timeout -- Timeout (in seconds) for startup of the pod (if pod is pending for too long, fails task)

Returns

follow_container_logs(self, pod: kubernetes.client.models.v1_pod.V1Pod, container_name: str) → None[source]¶: Follows the logs of container and streams to airflow logging. Returns when container exits.

Note

read_pod_logs() follows the logs, so we shouldn't necessarily need to loop as we do here. But in a long-running process we might temporarily lose connectivity. So the looping logic is there to let us resume following the logs.

await_container_completion(self, pod: kubernetes.client.models.v1_pod.V1Pod, container_name: str) → None[source]¶

await_pod_completion(self, pod: kubernetes.client.models.v1_pod.V1Pod) → kubernetes.client.models.v1_pod.V1Pod[source]¶

Monitors a pod and returns the final state

Parameters: pod -- pod spec that will be monitored
Returns: Tuple[State, Optional[str]]

parse_log_line(self, line: str) → Tuple[Optional[Union[pendulum.Date, pendulum.Time, pendulum.DateTime, pendulum.Duration]], str][source]¶

Parse K8s log line and returns the final state

Parameters: line (str) -- k8s log line
Returns: timestamp and log message
Return type: Tuple[str, str]

container_is_running(self, pod: kubernetes.client.models.v1_pod.V1Pod, container_name: str) → bool[source]¶: Reads pod and checks if container is running

read_pod_logs(self, pod: kubernetes.client.models.v1_pod.V1Pod, container_name: str, tail_lines: Optional[int] = None, timestamps: bool = False, since_seconds: Optional[int] = None) → Iterable[str][source]¶: Reads log from the POD

read_pod_events(self, pod: kubernetes.client.models.v1_pod.V1Pod) → kubernetes.client.models.core_v1_event_list.CoreV1EventList[source]¶: Reads events from the POD

read_pod(self, pod: kubernetes.client.models.v1_pod.V1Pod) → kubernetes.client.models.v1_pod.V1Pod[source]¶: Read POD information

extract_xcom(self, pod: kubernetes.client.models.v1_pod.V1Pod) → str[source]¶: Retrieves XCom value and kills xcom sidecar container

airflow.providers.cncf.kubernetes.utils.pod_manager¶

Module Contents¶

Classes¶

Functions¶

`airflow.providers.cncf.kubernetes.utils.pod_manager`¶