airflow.providers.snowflake.transfers.s3_to_snowflake

This module contains AWS S3 to Snowflake operator.

Module Contents

Classes

S3ToSnowflakeOperator

Executes an COPY command to load files from s3 to Snowflake
class airflow.providers.snowflake.transfers.s3_to_snowflake.S3ToSnowflakeOperator(*, s3_keys=None, table, stage, prefix=None, file_format, schema=None, columns_array=None, pattern=None, warehouse=None, database=None, autocommit=True, snowflake_conn_id='snowflake_default', role=None, authenticator=None, session_parameters=None, **kwargs)[source]

Bases: airflow.models.BaseOperator

Executes an COPY command to load files from s3 to Snowflake

See also

For more information on how to use this operator, take a look at the guide: S3ToSnowflakeOperator

Parameters

  • s3_keys (Optional[list]) – reference to a list of S3 keys

  • table (str) – reference to a specific table in snowflake database

  • schema (Optional[str]) – name of schema (will overwrite schema defined in connection)

  • stage (str) – reference to a specific snowflake stage. If the stage’s schema is not the same as the table one, it must be specified

  • prefix (Optional[str]) – cloud storage location specified to limit the set of files to load

  • file_format (str) – reference to a specific file format

  • warehouse (Optional[str]) – name of warehouse (will overwrite any warehouse defined in the connection’s extra JSON)

  • database (Optional[str]) – reference to a specific database in Snowflake connection

  • columns_array (Optional[list]) – reference to a specific columns array in snowflake database

  • pattern (Optional[str]) – regular expression pattern string specifying the file names and/or paths to match. Note: regular expression will be automatically enclose in single quotes and all single quotes in expression will replace by two single quotes.

  • snowflake_conn_id (str) – Reference to Snowflake connection id

  • role (Optional[str]) – name of role (will overwrite any role defined in connection’s extra JSON)

  • authenticator (Optional[str]) – authenticator for Snowflake. ‘snowflake’ (default) to use the internal Snowflake authenticator ‘externalbrowser’ to authenticate using your web browser and Okta, ADFS or any other SAML 2.0-compliant identify provider (IdP) that has been defined for your account ‘https://<your_okta_account_name>.okta.com’ to authenticate through native Okta.

  • session_parameters (Optional[dict]) – You can set session-level parameters at the time you connect to Snowflake

template_fields :Sequence[str] = ['s3_keys'][source]
template_fields_renderers[source]
execute(self, context)[source]

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

Was this entry helpful?