airflow.providers.google.cloud.triggers.cloud_storage_transfer_service¶

Classes¶

`CloudStorageTransferServiceCreateJobsTrigger`	StorageTransferJobTrigger run on the trigger worker to perform Cloud Storage Transfer job.
`CloudStorageTransferServiceCheckJobStatusTrigger`	CloudStorageTransferServiceCheckJobStatusTrigger run on the trigger worker to check Cloud Storage Transfer job.
`CloudDataTransferServiceRunJobTrigger`	CloudDataTransferServiceRunJobTrigger run on the trigger worker to run Cloud Storage Transfer job.

Module Contents¶

class airflow.providers.google.cloud.triggers.cloud_storage_transfer_service.CloudStorageTransferServiceCreateJobsTrigger(job_names, project_id=PROVIDE_PROJECT_ID, poll_interval=10, gcp_conn_id='google_cloud_default')[source]¶

Bases: airflow.triggers.base.BaseTrigger

StorageTransferJobTrigger run on the trigger worker to perform Cloud Storage Transfer job.

Parameters:

job_names (list[str]) – List of transfer jobs names.
project_id (str) – GCP project id.
poll_interval (int) – Interval in seconds between polls.
gcp_conn_id (str) – The connection ID used to connect to Google Cloud.

project_id = None[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

job_names[source]¶

poll_interval = 10[source]¶

serialize()[source]¶

Serialize StorageTransferJobsTrigger arguments and classpath.

async run()[source]¶

Get current data storage transfer jobs and yields a TriggerEvent.

get_async_hook()[source]¶

class airflow.providers.google.cloud.triggers.cloud_storage_transfer_service.CloudStorageTransferServiceCheckJobStatusTrigger(job_name, expected_statuses=None, project_id=PROVIDE_PROJECT_ID, poke_interval=10.0, gcp_conn_id='google_cloud_default', impersonation_chain=None)[source]¶

Bases: airflow.triggers.base.BaseTrigger

CloudStorageTransferServiceCheckJobStatusTrigger run on the trigger worker to check Cloud Storage Transfer job.

Parameters:

job_name (str) – The name of the transfer job
expected_statuses (set[str] | str | None) – The expected state of the operation. See: https://cloud.google.com/storage-transfer/docs/reference/rest/v1/transferOperations#Status
project_id (str) – The ID of the project that owns the Transfer Job.
poke_interval (float) – Polling period in seconds to check for the status
gcp_conn_id (str) – The connection ID used to connect to Google Cloud.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

job_name[source]¶

expected_statuses = None[source]¶

project_id = None[source]¶

poke_interval = 10.0[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

serialize()[source]¶

Serialize CloudStorageTransferServiceCheckJobStatusTrigger arguments and classpath.

async run()[source]¶

Check the status of the transfer job and yield a TriggerEvent.

class airflow.providers.google.cloud.triggers.cloud_storage_transfer_service.CloudDataTransferServiceRunJobTrigger(job_name, project_id=PROVIDE_PROJECT_ID, poke_interval=10.0, gcp_conn_id='google_cloud_default', impersonation_chain=None)[source]¶

Bases: airflow.triggers.base.BaseTrigger

CloudDataTransferServiceRunJobTrigger run on the trigger worker to run Cloud Storage Transfer job.

Parameters:

job_name (str) – The name of the transfer job
project_id (str) – The ID of the project that owns the Transfer Job.
poke_interval (float) – Polling period in seconds to check for the status
gcp_conn_id (str) – The connection ID used to connect to Google Cloud.
impersonation_chain (str | collections.abc.Sequence[str] | None) – Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

job_name[source]¶

project_id = None[source]¶

poke_interval = 10.0[source]¶

gcp_conn_id = 'google_cloud_default'[source]¶

impersonation_chain = None[source]¶

serialize()[source]¶

Serialize CloudDataTransferServiceRunJobTrigger arguments and classpath.

async run()[source]¶

Run the transfer job and yield a TriggerEvent.