airflow.contrib.operators.s3_copy_object_operator

Module Contents

class airflow.contrib.operators.s3_copy_object_operator.S3CopyObjectOperator(source_bucket_key, dest_bucket_key, source_bucket_name=None, dest_bucket_name=None, source_version_id=None, aws_conn_id='aws_default', verify=None, *args, **kwargs)[source]

Bases: airflow.models.BaseOperator

Creates a copy of an object that is already stored in S3.

Note: the S3 connection used here needs to have access to both source and destination bucket/key.

Parameters
  • source_bucket_key (str) –

    The key of the source object. (templated)

    It can be either full s3:// style url or relative path from root level.

    When it’s specified as a full s3:// url, please omit source_bucket_name.

  • dest_bucket_key (str) –

    The key of the object to copy to. (templated)

    The convention to specify dest_bucket_key is the same as source_bucket_key.

  • source_bucket_name (str) –

    Name of the S3 bucket where the source object is in. (templated)

    It should be omitted when source_bucket_key is provided as a full s3:// url.

  • dest_bucket_name (str) –

    Name of the S3 bucket to where the object is copied. (templated)

    It should be omitted when dest_bucket_key is provided as a full s3:// url.

  • source_version_id (str) – Version ID of the source object (OPTIONAL)

  • aws_conn_id (str) – Connection id of the S3 connection to use

  • verify (bool or str) –

    Whether or not to verify SSL certificates for S3 connection. By default SSL certificates are verified.

    You can provide the following values:

    • False: do not validate SSL certificates. SSL will still be used,

      but SSL certificates will not be verified.

    • path/to/cert/bundle.pem: A filename of the CA cert bundle to uses.

      You can specify this argument if you want to use a different CA cert bundle than the one used by botocore.

template_fields = ['source_bucket_key', 'dest_bucket_key', 'source_bucket_name', 'dest_bucket_name'][source]
execute(self, context)[source]

Was this entry helpful?