airflow.executors.dask_executor

DaskExecutor

See also

For more information on how the DaskExecutor works, take a look at the guide: Dask Executor

Module Contents

Classes

DaskExecutor

DaskExecutor submits tasks to a Dask Distributed cluster.
class airflow.executors.dask_executor.DaskExecutor(cluster_address=None)[source]

Bases: airflow.executors.base_executor.BaseExecutor

DaskExecutor submits tasks to a Dask Distributed cluster.

start(self)[source]

Executors may need to get things started.

execute_async(self, key, command, queue=None, executor_config=None)[source]

This method will execute the command asynchronously.

Parameters

  • key (airflow.models.taskinstance.TaskInstanceKey) -- Unique key for the task instance

  • command (airflow.executors.base_executor.CommandType) -- Command to run

  • queue (Optional[str]) -- name of the queue

  • executor_config (Optional[Any]) -- Configuration passed to the executor.

sync(self)[source]

Sync will get called periodically by the heartbeat method. Executors should override this to perform gather statuses.

end(self)[source]

This method is called when the caller is done submitting job and wants to wait synchronously for the job submitted previously to be all done.

terminate(self)[source]

This method is called when the daemon receives a SIGTERM

Was this entry helpful?