Microsoft Azure Data Factory Connection¶

The Microsoft Azure Data Factory connection type enables the Azure Data Factory Integrations.

Authenticating to Azure Data Factory¶

There is one way to connect to Azure Data Factory using Airflow.

Use token credentials i.e. add specific credentials (client_id, secret, tenant) and subscription id to the Airflow connection.

Default Connection IDs¶

All hooks and operators related to Microsoft Azure Data Factory use azure_data_factory_default by default.

Configuring the Connection¶

Login

Specify the client_id used for the initial connection. This is needed for token credentials authentication mechanism.

Password

Specify the secret used for the initial connection. This is needed for token credentials authentication mechanism.

Extra (optional)

Specify the extra parameters (as json dictionary) that can be used in Azure Data Lake connection. The following parameters are all optional:

tenantId: Specify the tenant to use. This is needed for token credentials authentication mechanism.
subscriptionId: Specify the subscription id to use. This is needed for token credentials authentication mechanism.
resourceGroup: Specify the azure resource group name.
factory: Specify the azure data factory to use

When specifying the connection in environment variable you should specify it using URI syntax.

Note that all components of the URI should be URL-encoded.

For example:

export AIRFLOW_CONN_AZURE_DATA_FACTORY_DEFAULT='azure-data-factory://client%20id:secret@?tenantId=tenant+id&subscriptionId=subscription+id&resourceGroup=group+name&factory=factory+name'