airflow.providers.google.cloud.triggers.cloud_storage_transfer_service

Classes

CloudStorageTransferServiceCreateJobsTrigger

StorageTransferJobTrigger run on the trigger worker to perform Cloud Storage Transfer job.

CloudStorageTransferServiceCheckJobStatusTrigger

CloudStorageTransferServiceCheckJobStatusTrigger run on the trigger worker to check Cloud Storage Transfer job.

Module Contents

class airflow.providers.google.cloud.triggers.cloud_storage_transfer_service.CloudStorageTransferServiceCreateJobsTrigger(job_names, project_id=PROVIDE_PROJECT_ID, poll_interval=10, gcp_conn_id='google_cloud_default')[source]

Bases: airflow.triggers.base.BaseTrigger

StorageTransferJobTrigger run on the trigger worker to perform Cloud Storage Transfer job.

Parameters:

  • job_names (list[str]) – List of transfer jobs names.

  • project_id (str) – GCP project id.

  • poll_interval (int) – Interval in seconds between polls.

  • gcp_conn_id (str) – The connection ID used to connect to Google Cloud.

project_id = None[source]
gcp_conn_id = 'google_cloud_default'[source]
job_names[source]
poll_interval = 10[source]
serialize()[source]

Serialize StorageTransferJobsTrigger arguments and classpath.

async run()[source]

Get current data storage transfer jobs and yields a TriggerEvent.

get_async_hook()[source]
class airflow.providers.google.cloud.triggers.cloud_storage_transfer_service.CloudStorageTransferServiceCheckJobStatusTrigger(job_name, expected_statuses=None, project_id=PROVIDE_PROJECT_ID, poke_interval=10.0, gcp_conn_id='google_cloud_default', impersonation_chain=None)[source]

Bases: airflow.triggers.base.BaseTrigger

CloudStorageTransferServiceCheckJobStatusTrigger run on the trigger worker to check Cloud Storage Transfer job.

Parameters:

  • job_name (str) – The name of the transfer job

  • expected_statuses (set[str] | str | None) – The expected state of the operation. See: https://cloud.google.com/storage-transfer/docs/reference/rest/v1/transferOperations#Status

  • project_id (str) – The ID of the project that owns the Transfer Job.

  • poke_interval (float) – Polling period in seconds to check for the status

  • gcp_conn_id (str) – The connection ID used to connect to Google Cloud.

  • impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

job_name[source]
expected_statuses = None[source]
project_id = None[source]
poke_interval = 10.0[source]
gcp_conn_id = 'google_cloud_default'[source]
impersonation_chain = None[source]
serialize()[source]

Serialize CloudStorageTransferServiceCheckJobStatusTrigger arguments and classpath.

async run()[source]

Check the status of the transfer job and yield a TriggerEvent.

Was this entry helpful?