airflow.providers.teradata.transfers.azure_blob_to_teradata¶

Classes¶

AzureBlobStorageToTeradataOperator

Loads CSV, JSON and Parquet format data from Azure Blob Storage to Teradata.

Module Contents¶

class airflow.providers.teradata.transfers.azure_blob_to_teradata.AzureBlobStorageToTeradataOperator(*, blob_source_key, public_bucket=False, azure_conn_id='azure_default', teradata_table, teradata_conn_id='teradata_default', teradata_authorization_name='', **kwargs)[source]¶

Bases: airflow.providers.common.compat.sdk.BaseOperator

Loads CSV, JSON and Parquet format data from Azure Blob Storage to Teradata.

See also

For more information on how to use this operator, take a look at the guide: AzureBlobStorageToTeradataOperator

Parameters:

blob_source_key (str) – The URI format specifying the location of the Azure blob object store.(templated) The URI format is /az/YOUR-STORAGE-ACCOUNT.blob.core.windows.net/YOUR-CONTAINER/YOUR-BLOB-LOCATION. Refer to https://docs.teradata.com/search/documents?query=native+object+store&sort=last_update&virtual-field=title_only&content-lang=en-US
public_bucket (bool) – Specifies whether the provided blob container is public. If the blob container is public, it means that anyone can access the objects within it via a URL without requiring authentication. If the bucket is private and authentication is not provided, the operator will throw an exception.
azure_conn_id (str) – The Airflow WASB connection used for azure blob credentials.
teradata_table (str) – The name of the teradata table to which the data is transferred.(templated)
teradata_conn_id (str) – The connection ID used to connect to Teradata Teradata connection
teradata_authorization_name (str) – The name of Teradata Authorization Database Object, is used to control who can access an Azure Blob object store. Refer to https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/Teradata-VantageTM-Native-Object-Store-Getting-Started-Guide-17.20/Setting-Up-Access/Controlling-Foreign-Table-Access-with-an-AUTHORIZATION-Object

When teradata_authorization_name is not set and the object store is private, the store’s credentials are sent to Teradata inside the CREATE TABLE statement itself. Teradata records that statement in its own query logs (DBQL) and monitoring views, which Airflow cannot redact, so the credentials are readable by anyone with access to those. Prefer an AUTHORIZATION object, which keeps the credentials in the database instead of in each statement. Note that blob_source_key and teradata_table are templated, so you can use variables in them if you wish.

template_fields: collections.abc.Sequence[str] = ('blob_source_key', 'teradata_table', 'azure_conn_id', 'teradata_conn_id')[source]¶

ui_color = '#e07c24'[source]¶

blob_source_key[source]¶

public_bucket = False[source]¶

azure_conn_id = 'azure_default'[source]¶

teradata_table[source]¶

teradata_conn_id = 'teradata_default'[source]¶

teradata_authorization_name = ''[source]¶

execute(context)[source]¶

Derive when creating an operator.

The main method to execute the task. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.