Amazon Glacier Transfer Operator¶

Amazon Glacier is a secure, durable, and extremely low-cost Amazon S3 cloud storage classes for data archiving and long-term backup. For more information about the service visit Amazon Glacier API documentation

GlacierToGCSOperator¶

Operator task is transfer data from Glacier vault to Google Cloud Storage.

Note

Please be aware that GlacierToGCSOperator may depend on memory usage. Transferring big files may not work well.

To get more information about operator visit: GlacierToGCSOperator

Example usage:

airflow/providers/amazon/aws/example_dags/example_glacier_to_gcs.pyView Source

transfer_archive_to_gcs = GlacierToGCSOperator(
    task_id="transfer_archive_to_gcs",
    aws_conn_id="aws_default",
    gcp_conn_id="google_cloud_default",
    vault_name=VAULT_NAME,
    bucket_name=BUCKET_NAME,
    object_name=OBJECT_NAME,
    gzip=False,
    # Override to match your needs
    # If chunk size is bigger than actual file size
    # then whole file will be downloaded
    chunk_size=1024,
    delegate_to=None,
    google_impersonation_chain=None,
)