CeleryKubernetesExecutor consists of CeleryExecutor and KubernetesExecutor.
- class airflow.executors.celery_kubernetes_executor.CeleryKubernetesExecutor(celery_executor, kubernetes_executor)¶
CeleryKubernetesExecutor consists of CeleryExecutor and KubernetesExecutor. It chooses an executor to use based on the queue defined on the task. When the queue is the value of
[celery_kubernetes_executor]of the configuration (default value: kubernetes), KubernetesExecutor is selected to run the task, otherwise, CeleryExecutor is used.
- supports_ad_hoc_ti_run :bool = True¶
- callback_sink :Optional[airflow.callbacks.base_callback_sink.BaseCallbackSink]¶
- property queued_tasks(self)¶
Return queued tasks from celery and kubernetes executor
- property running(self)¶
Return running tasks from celery and kubernetes executor
- property job_id(self)¶
This is a class attribute in BaseExecutor but since this is not really an executor, but a wrapper of executors we implement as property so we can have custom setter.
Start celery and kubernetes executor
- property slots_available(self)¶
Number of new tasks this executor instance can accept
- queue_command(self, task_instance, command, priority=1, queue=None)¶
Queues command via celery or kubernetes executor
- queue_task_instance(self, task_instance, mark_success=False, pickle_id=None, ignore_all_deps=False, ignore_depends_on_past=False, ignore_task_deps=False, ignore_ti_state=False, pool=None, cfg_path=None)¶
Queues task instance via celery or kubernetes executor
- has_task(self, task_instance)¶
Checks if a task is either queued or running in either celery or kubernetes executor.
Heartbeat sent to trigger new jobs in celery and kubernetes executor
- get_event_buffer(self, dag_ids=None)¶
Returns and flush the event buffer from celery and kubernetes executor
- try_adopt_task_instances(self, tis)¶
Try to adopt running task instances that have been abandoned by a SchedulerJob dying.
Anything that is not adopted will be cleared by the scheduler (and then become eligible for re-scheduling)
any TaskInstances that were unable to be adopted
- Return type
End celery and kubernetes executor
Terminate celery and kubernetes executor
Called in response to SIGUSR2 by the scheduler
- send_callback(self, request)¶
Sends callback for execution.
request (airflow.callbacks.callback_requests.CallbackRequest) – Callback request to be executed.