airflow.providers.cncf.kubernetes.kubernetes_helper_functions

Module Contents

Functions

create_unique_id([dag_id, task_id, max_length, unique])

Generate unique pod or job ID given a dag_id and / or task_id.

create_pod_id([dag_id, task_id, max_length, unique])

Generate unique pod ID given a dag_id and / or task_id.

annotations_to_key(annotations)

Build a TaskInstanceKey based on pod annotations.

get_logs_task_metadata()

annotations_for_logging_task_metadata(annotation_set)

should_retry_creation(exception)

Check if an Exception indicates a transient error and warrants retrying.

Attributes

log

alphanum_lower

POD_NAME_MAX_LENGTH

airflow.providers.cncf.kubernetes.kubernetes_helper_functions.log[source]
airflow.providers.cncf.kubernetes.kubernetes_helper_functions.alphanum_lower[source]
airflow.providers.cncf.kubernetes.kubernetes_helper_functions.POD_NAME_MAX_LENGTH = 63[source]
airflow.providers.cncf.kubernetes.kubernetes_helper_functions.create_unique_id(dag_id=None, task_id=None, *, max_length=POD_NAME_MAX_LENGTH, unique=True)[source]

Generate unique pod or job ID given a dag_id and / or task_id.

Parameters

  • dag_id (str | None) – DAG ID

  • task_id (str | None) – Task ID

  • max_length (int) – max number of characters

  • unique (bool) – whether a random string suffix should be added

Returns

A valid identifier for a kubernetes pod name

Return type

str

airflow.providers.cncf.kubernetes.kubernetes_helper_functions.create_pod_id(dag_id=None, task_id=None, *, max_length=POD_NAME_MAX_LENGTH, unique=True)[source]

Generate unique pod ID given a dag_id and / or task_id.

Parameters

  • dag_id (str | None) – DAG ID

  • task_id (str | None) – Task ID

  • max_length (int) – max number of characters

  • unique (bool) – whether a random string suffix should be added

Returns

A valid identifier for a kubernetes pod name

Return type

str

airflow.providers.cncf.kubernetes.kubernetes_helper_functions.annotations_to_key(annotations)[source]

Build a TaskInstanceKey based on pod annotations.

airflow.providers.cncf.kubernetes.kubernetes_helper_functions.get_logs_task_metadata()[source]
airflow.providers.cncf.kubernetes.kubernetes_helper_functions.annotations_for_logging_task_metadata(annotation_set)[source]
airflow.providers.cncf.kubernetes.kubernetes_helper_functions.should_retry_creation(exception)[source]

Check if an Exception indicates a transient error and warrants retrying.

This function is needed for preventing ‘No agent available’ error. The error appears time to time when users try to create a Resource or Job. This issue is inside kubernetes and in the current moment has no solution. Like a temporary solution we decided to retry Job or Resource creation request each time when this error appears. More about this issue here: https://github.com/cert-manager/cert-manager/issues/6457

Was this entry helpful?