airflow.providers.amazon.aws.transfers.s3_to_redshift
¶
Module Contents¶
Classes¶
Executes an COPY command to load files from s3 to Redshift |
Attributes¶
- airflow.providers.amazon.aws.transfers.s3_to_redshift.AVAILABLE_METHODS = ['APPEND', 'REPLACE', 'UPSERT'][source]¶
- class airflow.providers.amazon.aws.transfers.s3_to_redshift.S3ToRedshiftOperator(*, schema, table, s3_bucket, s3_key, redshift_conn_id='redshift_default', aws_conn_id='aws_default', verify=None, column_list=None, copy_options=None, autocommit=False, method='APPEND', upsert_keys=None, **kwargs)[source]¶
Bases:
airflow.models.BaseOperator
Executes an COPY command to load files from s3 to Redshift
See also
For more information on how to use this operator, take a look at the guide: Amazon S3 To Amazon Redshift transfer operator
- Parameters
schema (str) – reference to a specific schema in redshift database
table (str) – reference to a specific table in redshift database
s3_bucket (str) – reference to a specific S3 bucket
s3_key (str) – reference to a specific S3 key
redshift_conn_id (str) – reference to a specific redshift database
aws_conn_id (str) – reference to a specific S3 connection If the AWS connection contains ‘aws_iam_role’ in
extras
the operator will use AWS STS credentials with a token https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-authorization.html#copy-credentialsverify (Optional[Union[bool, str]]) –
Whether or not to verify SSL certificates for S3 connection. By default SSL certificates are verified. You can provide the following values:
False
: do not validate SSL certificates. SSL will still be used(unless use_ssl is False), but SSL certificates will not be verified.
path/to/cert/bundle.pem
: A filename of the CA cert bundle to uses.You can specify this argument if you want to use a different CA cert bundle than the one used by botocore.
column_list (Optional[List[str]]) – list of column names to load
copy_options (Optional[List]) – reference to a list of COPY options
method (str) – Action to be performed on execution. Available
APPEND
,UPSERT
andREPLACE
.upsert_keys (Optional[List[str]]) – List of fields to use as key on upsert action