airflow.executors.dask_executor

DaskExecutor

See also

For more information on how the DaskExecutor works, take a look at the guide: Dask Executor

Module Contents

Classes

DaskExecutor

DaskExecutor submits tasks to a Dask Distributed cluster.
class airflow.executors.dask_executor.DaskExecutor(cluster_address=None)[source]

Bases: airflow.executors.base_executor.BaseExecutor

DaskExecutor submits tasks to a Dask Distributed cluster.

start(self) → None[source]

Executors may need to get things started.

execute_async(self, key: airflow.models.taskinstance.TaskInstanceKey, command: airflow.executors.base_executor.CommandType, queue: Optional[str] = None, executor_config: Optional[Any] = None) → None[source]

This method will execute the command asynchronously.

Parameters

  • key – Unique key for the task instance

  • command – Command to run

  • queue – name of the queue

  • executor_config – Configuration passed to the executor.

sync(self) → None[source]

Sync will get called periodically by the heartbeat method. Executors should override this to perform gather statuses.

end(self) → None[source]

This method is called when the caller is done submitting job and wants to wait synchronously for the job submitted previously to be all done.

terminate(self)[source]

This method is called when the daemon receives a SIGTERM

Was this entry helpful?