airflow.executors.local_executor

LocalExecutor

See also

For more information on how the LocalExecutor works, take a look at the guide: Local Executor

Module Contents

class airflow.executors.local_executor.LocalWorker(result_queue)[source]

Bases: multiprocessing.Process, airflow.utils.log.logging_mixin.LoggingMixin

LocalWorker Process implementation to run airflow commands. Executes the given command and puts the result into a result queue when done, terminating execution.

execute_work(self, key, command)[source]

Executes command received and stores result state in queue.

Parameters

  • key (tuple(dag_id, task_id, execution_date)) – the key to identify the TI

  • command (str) – the command to execute

run(self)[source]
class airflow.executors.local_executor.QueuedLocalWorker(task_queue, result_queue)[source]

Bases: airflow.executors.local_executor.LocalWorker

LocalWorker implementation that is waiting for tasks from a queue and will continue executing commands as they become available in the queue. It will terminate execution once the poison token is found.

run(self)[source]
class airflow.executors.local_executor.LocalExecutor[source]

Bases: airflow.executors.base_executor.BaseExecutor

LocalExecutor executes tasks locally in parallel. It uses the multiprocessing Python library and queues to parallelize the execution of tasks.

class _UnlimitedParallelism(executor)[source]

Bases: object

Implements LocalExecutor with unlimited parallelism, starting one process per each command to execute.

start(self)[source]
execute_async(self, key, command)[source]
Parameters

  • key (tuple(dag_id, task_id, execution_date)) – the key to identify the TI

  • command (str) – the command to execute

sync(self)[source]
end(self)[source]
class _LimitedParallelism(executor)[source]

Bases: object

Implements LocalExecutor with limited parallelism using a task queue to coordinate work distribution.

start(self)[source]
execute_async(self, key, command)[source]
Parameters

  • key (tuple(dag_id, task_id, execution_date)) – the key to identify the TI

  • command (str) – the command to execute

sync(self)[source]
end(self)[source]
start(self)[source]
execute_async(self, key, command, queue=None, executor_config=None)[source]
sync(self)[source]
end(self)[source]

Was this entry helpful?