Transfers data from Azure FileShare Storage to Google Cloud Storage¶

The Google Cloud Storage (GCS) is used to store large data from various applications. The Azure FileShare (Azure FileShare) is very similar to GCS but from another provider (Azure). This page shows how to transfer data from Azure FileShare to GCS.

Overview¶

Data transfer¶

Transfer of files between Azure FileShare and Google Storage is performed with the AzureFileShareToGCSOperator operator.

This operator has 1 required parameter:

share_name - The Azure FileShare share name to transfer files from.

All parameters are described in the reference documentation - AzureFileShareToGCSOperator.

An example operator call might look like this:

airflow/providers/google/cloud/example_dags/example_azure_fileshare_to_gcs.pyView Source

sync_azure_files_with_gcs = AzureFileShareToGCSOperator(
    task_id='sync_azure_files_with_gcs',
    share_name=AZURE_SHARE_NAME,
    dest_gcs=DEST_GCS_BUCKET,
    directory_name=AZURE_DIRECTORY_NAME,
    wasb_conn_id='azure_fileshare_default',
    gcp_conn_id='google_cloud_default',
    replace=False,
    gzip=True,
    google_impersonation_chain=None,
)

Reference¶

For further information, look at: