`airflow.executors.base_executor`¶

Module Contents¶

airflow.executors.base_executor.PARALLELISM[source]¶

class airflow.executors.base_executor.BaseExecutor(parallelism=PARALLELISM)[source]¶

Bases: airflow.utils.log.logging_mixin.LoggingMixin

start(self)[source]¶: Executors may need to get things started. For example LocalExecutor starts N workers.

queue_command(self, simple_task_instance, command, priority=1, queue=None)[source]¶

queue_task_instance(self, task_instance, mark_success=False, pickle_id=None, ignore_all_deps=False, ignore_depends_on_past=False, ignore_task_deps=False, ignore_ti_state=False, pool=None, cfg_path=None)[source]¶

has_task(self, task_instance)[source]¶

Checks if a task is either queued or running in this executor

Parameters: task_instance – TaskInstance
Returns: True if the task is known to this executor

sync(self)[source]¶: Sync will get called periodically by the heartbeat method. Executors should override this to perform gather statuses.

heartbeat(self)[source]¶

trigger_tasks(self, open_slots)[source]¶

Trigger tasks

Parameters: open_slots – Number of open slots
Returns

change_state(self, key, state)[source]¶

fail(self, key)[source]¶

success(self, key)[source]¶

get_event_buffer(self, dag_ids=None)[source]¶

Returns and flush the event buffer. In case dag_ids is specified it will only return and flush events for the given dag_ids. Otherwise it returns and flushes all

Parameters: dag_ids – to dag_ids to return events for, if None returns all
Returns: a dict of events

execute_async(self, key, command, queue=None, executor_config=None)[source]¶: This method will execute the command asynchronously.

end(self)[source]¶: This method is called when the caller is done submitting job and is wants to wait synchronously for the job submitted previously to be all done.

terminate(self)[source]¶: This method is called when the daemon receives a SIGTERM

airflow.executors.base_executor¶

Module Contents¶

`airflow.executors.base_executor`¶