airflow.contrib.sensors.emr_step_sensor

Module Contents

class airflow.contrib.sensors.emr_step_sensor.EmrStepSensor(job_flow_id, step_id, *args, **kwargs)[source]

Bases:airflow.contrib.sensors.emr_base_sensor.EmrBaseSensor

Asks for the state of the step until it reaches a terminal state. If it fails the sensor errors, failing the task.

Parameters
  • job_flow_id (str) – job_flow_id which contains the step check the state of

  • step_id (str) – step to check the state of

NON_TERMINAL_STATES = ['PENDING', 'RUNNING', 'CONTINUE', 'CANCEL_PENDING'][source]
FAILED_STATE = ['CANCELLED', 'FAILED', 'INTERRUPTED'][source]
template_fields = ['job_flow_id', 'step_id'][source]
template_ext = [][source]
get_emr_response(self)[source]
static state_from_response(response)[source]