airflow.providers.microsoft.azure.transfers.azure_blob_to_gcs

Module Contents

Classes

AzureBlobStorageToGCSOperator

Operator transfers data from Azure Blob Storage to specified bucket in Google Cloud Storage

class airflow.providers.microsoft.azure.transfers.azure_blob_to_gcs.AzureBlobStorageToGCSOperator(*, wasb_conn_id='wasb_default', gcp_conn_id='google_cloud_default', blob_name, file_path, container_name, bucket_name, object_name, filename, gzip, delegate_to, impersonation_chain=None, **kwargs)[source]

Bases: airflow.models.BaseOperator

Operator transfers data from Azure Blob Storage to specified bucket in Google Cloud Storage

See also

For more information on how to use this operator, take a look at the guide: Transfer Data from Blob Storage to Google Cloud Storage

Parameters
  • wasb_conn_id -- Reference to the wasb connection.

  • gcp_conn_id (str) -- The connection ID to use when fetching connection info.

  • blob_name (str) -- Name of the blob

  • file_path (str) -- Path to the file to download

  • container_name (str) -- Name of the container

  • bucket_name (str) -- The bucket to upload to

  • object_name (str) -- The object name to set when uploading the file

  • filename (str) -- The local file path to the file to be uploaded

  • gzip (bool) -- Option to compress local file or file data for upload

  • delegate_to (Optional[str]) -- The account to impersonate using domain-wide delegation of authority, if any. For this to work, the service account making the request must have domain-wide delegation enabled.

  • impersonation_chain (Optional[Union[str, Sequence[str]]]) -- Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account.

template_fields :Sequence[str] = ['blob_name', 'file_path', 'container_name', 'bucket_name', 'object_name', 'filename'][source]
execute(self, context)[source]

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

Was this entry helpful?