airflow.providers.snowflake.transfers.s3_to_snowflake

This module contains AWS S3 to Snowflake operator.

Module Contents

class airflow.providers.snowflake.transfers.s3_to_snowflake.S3ToSnowflakeOperator(*, s3_keys: Optional[list] = None, table: str, stage: str, prefix: Optional[str] = None, file_format: str, schema: str, columns_array: Optional[list] = None, warehouse: Optional[str] = None, database: Optional[str] = None, autocommit: bool = True, snowflake_conn_id: str = 'snowflake_default', role: Optional[str] = None, authenticator: Optional[str] = None, session_parameters: Optional[dict] = None, **kwargs)[source]

Bases: airflow.models.BaseOperator

Executes an COPY command to load files from s3 to Snowflake

See also

For more information on how to use this operator, take a look at the guide: S3ToSnowflakeOperator

Parameters

  • s3_keys (list) -- reference to a list of S3 keys

  • table (str) -- reference to a specific table in snowflake database

  • schema (str) -- name of schema (will overwrite schema defined in connection)

  • stage (str) -- reference to a specific snowflake stage. If the stage's schema is not the same as the table one, it must be specified

  • prefix (str) -- cloud storage location specified to limit the set of files to load

  • file_format (str) -- reference to a specific file format

  • warehouse (str) -- name of warehouse (will overwrite any warehouse defined in the connection's extra JSON)

  • database (str) -- reference to a specific database in Snowflake connection

  • columns_array (list) -- reference to a specific columns array in snowflake database

  • snowflake_conn_id (str) -- Reference to Snowflake connection id

  • role (str) -- name of role (will overwrite any role defined in connection's extra JSON)

  • authenticator (str) -- authenticator for Snowflake. 'snowflake' (default) to use the internal Snowflake authenticator 'externalbrowser' to authenticate using your web browser and Okta, ADFS or any other SAML 2.0-compliant identify provider (IdP) that has been defined for your account 'https://<your_okta_account_name>.okta.com' to authenticate through native Okta.

  • session_parameters (dict) -- You can set session-level parameters at the time you connect to Snowflake

execute(self, context: Any)[source]

Was this entry helpful?