airflow.contrib.sensors.pubsub_sensor

Module Contents

class airflow.contrib.sensors.pubsub_sensor.PubSubPullSensor(project, subscription, max_messages=5, return_immediately=False, ack_messages=False, gcp_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs)

Bases: airflow.sensors.base_sensor_operator.BaseSensorOperator

Pulls messages from a PubSub subscription and passes them through XCom.

This sensor operator will pull up to max_messages messages from the specified PubSub subscription. When the subscription returns messages, the poke method’s criteria will be fulfilled and the messages will be returned from the operator and passed through XCom for downstream tasks.

If ack_messages is set to True, messages will be immediately acknowledged before being returned, otherwise, downstream tasks will be responsible for acknowledging them.

project and subscription are templated so you can use variables in them.

template_fields = ['project', 'subscription']

ui_color = #ff7f50

execute(self, context)

Overridden to allow messages to be passed

poke(self, context)