airflow.contrib.executors.mesos_executor

Module Contents

airflow.contrib.executors.mesos_executor.DEFAULT_FRAMEWORK_NAME = Airflow[source]
airflow.contrib.executors.mesos_executor.FRAMEWORK_CONNID_PREFIX = mesos_framework_[source]
airflow.contrib.executors.mesos_executor.get_framework_name()[source]
class airflow.contrib.executors.mesos_executor.AirflowMesosScheduler(task_queue, result_queue, task_cpu=1, task_mem=256)[source]

Bases: mesos.interface.Scheduler

Airflow Mesos scheduler implements mesos scheduler interface to schedule airflow tasks on mesos. Basically, it schedules a command like ‘airflow run <dag_id> <task_instance_id> <start_date> –local -p=<pickle>’ to run on a mesos slave.

registered(self, driver, frameworkId, masterInfo)[source]
reregistered(self, driver, masterInfo)[source]
disconnected(self, driver)[source]
offerRescinded(self, driver, offerId)[source]
frameworkMessage(self, driver, executorId, slaveId, message)[source]
executorLost(self, driver, executorId, slaveId, status)[source]
slaveLost(self, driver, slaveId)[source]
error(self, driver, message)[source]
resourceOffers(self, driver, offers)[source]
statusUpdate(self, driver, update)[source]
class airflow.contrib.executors.mesos_executor.MesosExecutor[source]

Bases: airflow.executors.base_executor.BaseExecutor

MesosExecutor allows distributing the execution of task instances to multiple mesos workers.

Apache Mesos is a distributed systems kernel which abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively. See http://mesos.apache.org/

start(self)[source]
execute_async(self, key, command, queue=None, executor_config=None)[source]
sync(self)[source]
end(self)[source]