:mod:`airflow.contrib.operators.pubsub_operator`
================================================

.. py:module:: airflow.contrib.operators.pubsub_operator







Module Contents
---------------






.. py:class:: PubSubTopicCreateOperator(project, topic, fail_if_exists=False, gcp_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs)

   Bases::class:`airflow.models.BaseOperator`

   

   Create a PubSub topic.

   By default, if the topic already exists, this operator will
   not cause the DAG to fail. ::

       with DAG('successful DAG') as dag:
           (
               dag
               >> PubSubTopicCreateOperator(project='my-project',
                                            topic='my_new_topic')
               >> PubSubTopicCreateOperator(project='my-project',
                                            topic='my_new_topic')
           )

   The operator can be configured to fail if the topic already exists. ::

       with DAG('failing DAG') as dag:
           (
               dag
               >> PubSubTopicCreateOperator(project='my-project',
                                            topic='my_new_topic')
               >> PubSubTopicCreateOperator(project='my-project',
                                            topic='my_new_topic',
                                            fail_if_exists=True)
           )

   Both ``project`` and ``topic`` are templated so you can use
   variables in them.


   

   

   .. attribute:: template_fields
      :annotation: = ['project', 'topic'] 

      



   

   

   .. attribute:: ui_color
      :annotation: = #0273d4 

      



   

   

   

   .. method:: execute(self, context)

      











.. py:class:: PubSubSubscriptionCreateOperator(topic_project, topic, subscription=None, subscription_project=None, ack_deadline_secs=10, fail_if_exists=False, gcp_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs)

   Bases::class:`airflow.models.BaseOperator`

   

   Create a PubSub subscription.

   By default, the subscription will be created in ``topic_project``. If
   ``subscription_project`` is specified and the GCP credentials allow, the
   Subscription can be created in a different project from its topic.

   By default, if the subscription already exists, this operator will
   not cause the DAG to fail. However, the topic must exist in the project. ::

       with DAG('successful DAG') as dag:
           (
               dag
               >> PubSubSubscriptionCreateOperator(
                   topic_project='my-project', topic='my-topic',
                   subscription='my-subscription')
               >> PubSubSubscriptionCreateOperator(
                   topic_project='my-project', topic='my-topic',
                   subscription='my-subscription')
           )

   The operator can be configured to fail if the subscription already exists.
   ::

       with DAG('failing DAG') as dag:
           (
               dag
               >> PubSubSubscriptionCreateOperator(
                   topic_project='my-project', topic='my-topic',
                   subscription='my-subscription')
               >> PubSubSubscriptionCreateOperator(
                   topic_project='my-project', topic='my-topic',
                   subscription='my-subscription', fail_if_exists=True)
           )

   Finally, subscription is not required. If not passed, the operator will
   generated a universally unique identifier for the subscription's name. ::

       with DAG('DAG') as dag:
           (
               dag >> PubSubSubscriptionCreateOperator(
                   topic_project='my-project', topic='my-topic')
           )

   ``topic_project``, ``topic``, ``subscription``, and
   ``subscription`` are templated so you can use variables in them.


   

   

   .. attribute:: template_fields
      :annotation: = ['topic_project', 'topic', 'subscription', 'subscription_project'] 

      



   

   

   .. attribute:: ui_color
      :annotation: = #0273d4 

      



   

   

   

   .. method:: execute(self, context)

      











.. py:class:: PubSubTopicDeleteOperator(project, topic, fail_if_not_exists=False, gcp_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs)

   Bases::class:`airflow.models.BaseOperator`

   

   Delete a PubSub topic.

   By default, if the topic does not exist, this operator will
   not cause the DAG to fail. ::

       with DAG('successful DAG') as dag:
           (
               dag
               >> PubSubTopicDeleteOperator(project='my-project',
                                            topic='non_existing_topic')
           )

   The operator can be configured to fail if the topic does not exist. ::

       with DAG('failing DAG') as dag:
           (
               dag
               >> PubSubTopicCreateOperator(project='my-project',
                                            topic='non_existing_topic',
                                            fail_if_not_exists=True)
           )

   Both ``project`` and ``topic`` are templated so you can use
   variables in them.


   

   

   .. attribute:: template_fields
      :annotation: = ['project', 'topic'] 

      



   

   

   .. attribute:: ui_color
      :annotation: = #cb4335 

      



   

   

   

   .. method:: execute(self, context)

      











.. py:class:: PubSubSubscriptionDeleteOperator(project, subscription, fail_if_not_exists=False, gcp_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs)

   Bases::class:`airflow.models.BaseOperator`

   

   Delete a PubSub subscription.

   By default, if the subscription does not exist, this operator will
   not cause the DAG to fail. ::

       with DAG('successful DAG') as dag:
           (
               dag
               >> PubSubSubscriptionDeleteOperator(project='my-project',
                                                   subscription='non-existing')
           )

   The operator can be configured to fail if the subscription already exists.

   ::

       with DAG('failing DAG') as dag:
           (
               dag
               >> PubSubSubscriptionDeleteOperator(
                    project='my-project', subscription='non-existing',
                    fail_if_not_exists=True)
           )

   ``project``, and ``subscription`` are templated so you can use
   variables in them.


   

   

   .. attribute:: template_fields
      :annotation: = ['project', 'subscription'] 

      



   

   

   .. attribute:: ui_color
      :annotation: = #cb4335 

      



   

   

   

   .. method:: execute(self, context)

      











.. py:class:: PubSubPublishOperator(project, topic, messages, gcp_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs)

   Bases::class:`airflow.models.BaseOperator`

   

   Publish messages to a PubSub topic.

   Each Task publishes all provided messages to the same topic
   in a single GCP project. If the topic does not exist, this
   task will fail. ::

       from base64 import b64encode as b64e

       m1 = {'data': b64e('Hello, World!'),
             'attributes': {'type': 'greeting'}
            }
       m2 = {'data': b64e('Knock, knock')}
       m3 = {'attributes': {'foo': ''}}

       t1 = PubSubPublishOperator(
           project='my-project',topic='my_topic',
           messages=[m1, m2, m3],
           create_topic=True,
           dag=dag)

   ``project`` , ``topic``, and ``messages`` are templated so you can use
   variables in them.


   

   

   .. attribute:: template_fields
      :annotation: = ['project', 'topic', 'messages'] 

      



   

   

   .. attribute:: ui_color
      :annotation: = #0273d4 

      



   

   

   

   .. method:: execute(self, context)