:mod:`airflow.contrib.hooks.gcp_pubsub_hook` ============================================ .. py:module:: airflow.contrib.hooks.gcp_pubsub_hook Module Contents --------------- .. function:: _format_subscription(project, subscription) .. function:: _format_topic(project, topic) .. py:exception:: PubSubException Bases: :class:`Exception` .. py:class:: PubSubHook(gcp_conn_id='google_cloud_default', delegate_to=None) Bases: :class:`airflow.contrib.hooks.gcp_api_base_hook.GoogleCloudBaseHook` Hook for accessing Google Pub/Sub. The GCP project against which actions are applied is determined by the project embedded in the Connection referenced by gcp_conn_id. .. method:: get_conn(self) Returns a Pub/Sub service object. :rtype: googleapiclient.discovery.Resource .. method:: publish(self, project, topic, messages) Publishes messages to a Pub/Sub topic. :param project: the GCP project ID in which to publish :type project: str :param topic: the Pub/Sub topic to which to publish; do not include the ``projects/{project}/topics/`` prefix. :type topic: str :param messages: messages to publish; if the data field in a message is set, it should already be base64 encoded. :type messages: list of PubSub messages; see http://cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage .. method:: create_topic(self, project, topic, fail_if_exists=False) Creates a Pub/Sub topic, if it does not already exist. :param project: the GCP project ID in which to create the topic :type project: str :param topic: the Pub/Sub topic name to create; do not include the ``projects/{project}/topics/`` prefix. :type topic: str :param fail_if_exists: if set, raise an exception if the topic already exists :type fail_if_exists: bool .. method:: delete_topic(self, project, topic, fail_if_not_exists=False) Deletes a Pub/Sub topic if it exists. :param project: the GCP project ID in which to delete the topic :type project: str :param topic: the Pub/Sub topic name to delete; do not include the ``projects/{project}/topics/`` prefix. :type topic: str :param fail_if_not_exists: if set, raise an exception if the topic does not exist :type fail_if_not_exists: bool .. method:: create_subscription(self, topic_project, topic, subscription=None, subscription_project=None, ack_deadline_secs=10, fail_if_exists=False) Creates a Pub/Sub subscription, if it does not already exist. :param topic_project: the GCP project ID of the topic that the subscription will be bound to. :type topic_project: str :param topic: the Pub/Sub topic name that the subscription will be bound to create; do not include the ``projects/{project}/subscriptions/`` prefix. :type topic: str :param subscription: the Pub/Sub subscription name. If empty, a random name will be generated using the uuid module :type subscription: str :param subscription_project: the GCP project ID where the subscription will be created. If unspecified, ``topic_project`` will be used. :type subscription_project: str :param ack_deadline_secs: Number of seconds that a subscriber has to acknowledge each message pulled from the subscription :type ack_deadline_secs: int :param fail_if_exists: if set, raise an exception if the topic already exists :type fail_if_exists: bool :return: subscription name which will be the system-generated value if the ``subscription`` parameter is not supplied :rtype: str .. method:: delete_subscription(self, project, subscription, fail_if_not_exists=False) Deletes a Pub/Sub subscription, if it exists. :param project: the GCP project ID where the subscription exists :type project: str :param subscription: the Pub/Sub subscription name to delete; do not include the ``projects/{project}/subscriptions/`` prefix. :type subscription: str :param fail_if_not_exists: if set, raise an exception if the topic does not exist :type fail_if_not_exists: bool .. method:: pull(self, project, subscription, max_messages, return_immediately=False) Pulls up to ``max_messages`` messages from Pub/Sub subscription. :param project: the GCP project ID where the subscription exists :type project: str :param subscription: the Pub/Sub subscription name to pull from; do not include the 'projects/{project}/topics/' prefix. :type subscription: str :param max_messages: The maximum number of messages to return from the Pub/Sub API. :type max_messages: int :param return_immediately: If set, the Pub/Sub API will immediately return if no messages are available. Otherwise, the request will block for an undisclosed, but bounded period of time :type return_immediately: bool :return: A list of Pub/Sub ReceivedMessage objects each containing an ``ackId`` property and a ``message`` property, which includes the base64-encoded message content. See https://cloud.google.com/pubsub/docs/reference/rest/v1/projects.subscriptions/pull#ReceivedMessage .. method:: acknowledge(self, project, subscription, ack_ids) Pulls up to ``max_messages`` messages from Pub/Sub subscription. :param project: the GCP project name or ID in which to create the topic :type project: str :param subscription: the Pub/Sub subscription name to delete; do not include the 'projects/{project}/topics/' prefix. :type subscription: str :param ack_ids: List of ReceivedMessage ackIds from a previous pull response :type ack_ids: list