airflow.providers.snowflake.transfers.s3_to_snowflake

This module contains AWS S3 to Snowflake operator.

Module Contents

Classes

S3ToSnowflakeOperator

Executes an COPY command to load files from s3 to Snowflake.
class airflow.providers.snowflake.transfers.s3_to_snowflake.S3ToSnowflakeOperator(*, s3_keys=None, table, stage, prefix=None, file_format, schema=None, columns_array=None, pattern=None, warehouse=None, database=None, autocommit=True, snowflake_conn_id='snowflake_default', role=None, authenticator=None, session_parameters=None, **kwargs)[source]

Bases: airflow.models.BaseOperator

Executes an COPY command to load files from s3 to Snowflake.

See also

For more information on how to use this operator, take a look at the guide: S3ToSnowflakeOperator

Parameters

  • s3_keys (list | None) – reference to a list of S3 keys

  • table (str) – reference to a specific table in snowflake database

  • schema (str | None) – name of schema (will overwrite schema defined in connection)

  • stage (str) – reference to a specific snowflake stage. If the stage’s schema is not the same as the table one, it must be specified

  • prefix (str | None) – cloud storage location specified to limit the set of files to load

  • file_format (str) – reference to a specific file format

  • warehouse (str | None) – name of warehouse (will overwrite any warehouse defined in the connection’s extra JSON)

  • database (str | None) – reference to a specific database in Snowflake connection

  • columns_array (list | None) – reference to a specific columns array in snowflake database

  • pattern (str | None) – regular expression pattern string specifying the file names and/or paths to match. Note: regular expression will be automatically enclose in single quotes and all single quotes in expression will replace by two single quotes.

  • snowflake_conn_id (str) – Reference to Snowflake connection id

  • role (str | None) – name of role (will overwrite any role defined in connection’s extra JSON)

  • authenticator (str | None) – authenticator for Snowflake. ‘snowflake’ (default) to use the internal Snowflake authenticator ‘externalbrowser’ to authenticate using your web browser and Okta, ADFS or any other SAML 2.0-compliant identify provider (IdP) that has been defined for your account ‘https://<your_okta_account_name>.okta.com’ to authenticate through native Okta.

  • session_parameters (dict | None) – You can set session-level parameters at the time you connect to Snowflake

template_fields: Sequence[str] = ('s3_keys',)[source]
template_fields_renderers[source]
execute(context)[source]

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

Was this entry helpful?