airflow.models.sensorinstance

Module Contents

Classes

SensorInstance

SensorInstance support the smart sensor service. It stores the sensor task states
class airflow.models.sensorinstance.SensorInstance(ti)[source]

Bases: airflow.models.base.Base

SensorInstance support the smart sensor service. It stores the sensor task states and context that required for poking include poke context and execution context. In sensor_instance table we also save the sensor operator classpath so that inside smart sensor there is no need to import the dagbag and create task object for each sensor task.

SensorInstance include another set of columns to support the smart sensor shard on large number of sensor instance. The key idea is to generate the hash code from the poke context and use it to map to a shorter shard code which can be used as an index. Every smart sensor process takes care of tasks whose shardcode are in a certain range.

__tablename__ = sensor_instance[source]
id[source]
task_id[source]
dag_id[source]
execution_date[source]
state[source]
start_date[source]
operator[source]
op_classpath[source]
hashcode[source]
shardcode[source]
poke_context[source]
execution_context[source]
created_at[source]
updated_at[source]
map_index[source]
__table_args__[source]
static get_classpath(obj)[source]

Get the object dotted class path. Used for getting operator classpath.

Parameters

obj --

Returns

The class path of input object

Return type

str

classmethod register(cls, ti, poke_context, execution_context, session=None)[source]

Register task instance ti for a sensor in sensor_instance table. Persist the context used for a sensor and set the sensor_instance table state to sensing.

Parameters

  • ti -- The task instance for the sensor to be registered.

  • poke_context -- Context used for sensor poke function.

  • execution_context -- Context used for execute sensor such as timeout setting and email configuration.

  • session -- SQLAlchemy ORM Session

Returns

True if the ti was registered successfully.

Return type

Boolean

property try_number(self)[source]

Return the try number that this task number will be when it is actually run. If the TI is currently running, this will match the column in the database, in all other cases this will be incremented.

__repr__(self)[source]
get_dagrun(self, session)[source]

Returns the DagRun for this SensorInstance

Parameters

session -- SQLAlchemy ORM Session

Returns

DagRun

Was this entry helpful?