Microsoft Azure

apache-airflow-providers-microsoft-azure

Microsoft Azure

Works with Airflow 2.11+

Install:

pip install apache-airflow-providers-microsoft-azure==13.0.1

Extras:

Version:

Airflow

2.11+

Python

>=3.10

Dependencies (20)

Show all Hide

apache-airflow>=2.11.0 apache-airflow-providers-common-compat>=1.13.0 adlfs>=2023.10.0 azure-batch>=8.0.0 azure-cosmos>=4.6.0 azure-mgmt-cosmosdb>=3.0.0 azure-datalake-store>=0.0.45 azure-identity>=1.3.1 azure-keyvault-secrets>=4.1.0 azure-mgmt-datalake-store>=0.5.0 azure-mgmt-resource>=2.2.0 azure-storage-blob>=12.26.0 azure-mgmt-storage>=16.0.0 azure-storage-file-share>=12.7.0 azure-servicebus>=7.12.1 azure-synapse-spark>=0.2.0 azure-synapse-artifacts>=0.17.0 azure-storage-file-datalake>=12.9.1 azure-kusto-data>=4.1.0,!=4.6.0,!=5.0.0 azure-mgmt-datafactory>=2.0.0

Connections (16)

Categories

Modules

O

ADLSCreateObjectOperator

Creates a new object from passed data to Azure Data Lake on specified file.

airflow.providers.microsoft.azure.operators.adls.ADLSCreateObjectOperator

View Docs Source

O

ADLSDeleteOperator

Delete files in the specified path.

airflow.providers.microsoft.azure.operators.adls.ADLSDeleteOperator

View Docs Source

O

ADLSListOperator

List all files from the specified path.

airflow.providers.microsoft.azure.operators.adls.ADLSListOperator

View Docs Source

O

AzureDataExplorerQueryOperator

Operator for querying Azure Data Explorer (Kusto).

airflow.providers.microsoft.azure.operators.adx.AzureDataExplorerQueryOperator

View Docs Source

O

AzureBatchOperator

Executes a job on Azure Batch Service.

airflow.providers.microsoft.azure.operators.batch.AzureBatchOperator

View Docs Source

O

AzureContainerInstancesOperator

Start a container on Azure Container Instances.

airflow.providers.microsoft.azure.operators.container_instances.AzureContainerInstancesOperator

View Docs Source

O

AzureCosmosInsertDocumentOperator

Insert a new document into the specified Cosmos database and collection.

airflow.providers.microsoft.azure.operators.cosmos.AzureCosmosInsertDocumentOperator

View Docs Source

O

WasbDeleteBlobOperator

Deletes blob(s) on Azure Blob Storage.

airflow.providers.microsoft.azure.operators.wasb_delete_blob.WasbDeleteBlobOperator

View Docs Source

O

AzureDataFactoryRunPipelineOperator

Execute a data factory pipeline.

airflow.providers.microsoft.azure.operators.data_factory.AzureDataFactoryRunPipelineOperator

View Docs Source

O

ASBReceiveSubscriptionMessageOperator

Receive a Batch messages from a Service Bus Subscription under specific Topic.

airflow.providers.microsoft.azure.operators.asb.ASBReceiveSubscriptionMessageOperator

View Docs Source

O

AzureServiceBusCreateQueueOperator

Create a Azure Service Bus queue under a Service Bus Namespace.

airflow.providers.microsoft.azure.operators.asb.AzureServiceBusCreateQueueOperator

View Docs Source

O

AzureServiceBusDeleteQueueOperator

Delete the Queue in the Azure Service Bus namespace.

airflow.providers.microsoft.azure.operators.asb.AzureServiceBusDeleteQueueOperator

View Docs Source

O

AzureServiceBusReceiveMessageOperator

Receive a batch of messages at once in a specified Queue name.

airflow.providers.microsoft.azure.operators.asb.AzureServiceBusReceiveMessageOperator

View Docs Source

O

AzureServiceBusSendMessageOperator

Send Message or batch message to the Service Bus queue.

airflow.providers.microsoft.azure.operators.asb.AzureServiceBusSendMessageOperator

View Docs Source

O

AzureServiceBusSubscriptionCreateOperator

Create an Azure Service Bus Topic Subscription under a Service Bus Namespace.

airflow.providers.microsoft.azure.operators.asb.AzureServiceBusSubscriptionCreateOperator

View Docs Source

O

AzureServiceBusSubscriptionDeleteOperator

Delete the topic subscription in the Azure ServiceBus namespace.

airflow.providers.microsoft.azure.operators.asb.AzureServiceBusSubscriptionDeleteOperator

View Docs Source

O

AzureServiceBusTopicCreateOperator

Create an Azure Service Bus Topic under a Service Bus Namespace.

airflow.providers.microsoft.azure.operators.asb.AzureServiceBusTopicCreateOperator

View Docs Source

O

AzureServiceBusTopicDeleteOperator

Delete the topic in the Azure Service Bus namespace.

airflow.providers.microsoft.azure.operators.asb.AzureServiceBusTopicDeleteOperator

View Docs Source

O

AzureServiceBusUpdateSubscriptionOperator

Update an Azure ServiceBus Topic Subscription under a ServiceBus Namespace.

airflow.providers.microsoft.azure.operators.asb.AzureServiceBusUpdateSubscriptionOperator

View Docs Source

O

AzureSynapseRunPipelineOperator

Execute a Synapse Pipeline.

airflow.providers.microsoft.azure.operators.synapse.AzureSynapseRunPipelineOperator

View Docs Source

O

AzureSynapseRunSparkBatchOperator

Execute a Spark job on Azure Synapse.

airflow.providers.microsoft.azure.operators.synapse.AzureSynapseRunSparkBatchOperator

View Docs Source

O

MSGraphAsyncOperator

A Microsoft Graph API operator which allows you to execute REST call to the Microsoft Graph API.

airflow.providers.microsoft.azure.operators.msgraph.MSGraphAsyncOperator

View Docs Source

O

PowerBIDatasetListOperator

Gets a list of datasets where the service principal from the connection is assigned as admin.

airflow.providers.microsoft.azure.operators.powerbi.PowerBIDatasetListOperator

View Docs Source

O

PowerBIDatasetRefreshOperator

Refreshes a Power BI dataset.

airflow.providers.microsoft.azure.operators.powerbi.PowerBIDatasetRefreshOperator

View Docs Source

O

PowerBIWorkspaceListOperator

Gets a list of workspaces where the service principal from the connection is assigned as admin.

airflow.providers.microsoft.azure.operators.powerbi.PowerBIWorkspaceListOperator

View Docs Source

H

AzureContainerVolumeHook

A hook which wraps an Azure Volume.

airflow.providers.microsoft.azure.hooks.container_volume.AzureContainerVolumeHook

View Docs Source

H

AzureContainerRegistryHook

A hook to communicate with a Azure Container Registry.

airflow.providers.microsoft.azure.hooks.container_registry.AzureContainerRegistryHook

View Docs Source

H

AzureContainerInstanceHook

A hook to communicate with Azure Container Instances.

airflow.providers.microsoft.azure.hooks.container_instance.AzureContainerInstanceHook

View Docs Source

H

AzureDataExplorerHook

Interact with Azure Data Explorer (Kusto).

airflow.providers.microsoft.azure.hooks.adx.AzureDataExplorerHook

View Docs Source

H

AzureFileShareHook

Interacts with Azure FileShare Storage.

airflow.providers.microsoft.azure.hooks.fileshare.AzureFileShareHook

View Docs Source

H

AzureBaseHook

This hook acts as a base hook for azure services.

airflow.providers.microsoft.azure.hooks.base_azure.AzureBaseHook

View Docs Source

H

AzureBatchHook

Hook for Azure Batch APIs.

airflow.providers.microsoft.azure.hooks.batch.AzureBatchHook

View Docs Source

H

AzureDataLakeHook

Integration with Azure Data Lake.

airflow.providers.microsoft.azure.hooks.data_lake.AzureDataLakeHook

View Docs Source

H

AzureDataLakeStorageV2Hook

Interact with a ADLS gen2 storage account.

airflow.providers.microsoft.azure.hooks.data_lake.AzureDataLakeStorageV2Hook

View Docs Source

H

AzureCosmosDBHook

Interact with Azure CosmosDB.

airflow.providers.microsoft.azure.hooks.cosmos.AzureCosmosDBHook

View Docs Source

H

WasbAsyncHook

An async hook that connects to Azure WASB to perform operations.

airflow.providers.microsoft.azure.hooks.wasb.WasbAsyncHook

View Docs Source

H

WasbHook

Interact with Azure Blob Storage through the ``wasb://`` protocol.

airflow.providers.microsoft.azure.hooks.wasb.WasbHook

View Docs Source

H

AzureDataFactoryAsyncHook

An Async Hook that connects to Azure DataFactory to perform pipeline operations.

airflow.providers.microsoft.azure.hooks.data_factory.AzureDataFactoryAsyncHook

View Docs Source

H

AzureDataFactoryHook

A hook to interact with Azure Data Factory.

airflow.providers.microsoft.azure.hooks.data_factory.AzureDataFactoryHook

View Docs Source

H

AdminClientHook

Interact with the ServiceBusAdministrationClient.

airflow.providers.microsoft.azure.hooks.asb.AdminClientHook

View Docs Source

H

MessageHook

Interact with ServiceBusClient.

airflow.providers.microsoft.azure.hooks.asb.MessageHook

View Docs Source

H

AzureSynapseHook

A hook to interact with Azure Synapse.

airflow.providers.microsoft.azure.hooks.synapse.AzureSynapseHook

View Docs Source

H

AzureSynapsePipelineHook

A hook to interact with Azure Synapse Pipeline.

airflow.providers.microsoft.azure.hooks.synapse.AzureSynapsePipelineHook

View Docs Source

H

KiotaRequestAdapterHook

A Microsoft Graph API interaction hook, a Wrapper around KiotaRequestAdapter.

airflow.providers.microsoft.azure.hooks.msgraph.KiotaRequestAdapterHook

View Docs Source

H

PowerBIHook

A async hook to interact with Power BI.

airflow.providers.microsoft.azure.hooks.powerbi.PowerBIHook

View Docs Source

S

AzureCosmosDocumentSensor

Checks for the existence of a document which matches the given query in CosmosDB.

airflow.providers.microsoft.azure.sensors.cosmos.AzureCosmosDocumentSensor

View Docs Source

S

WasbBlobSensor

Waits for a blob to arrive on Azure Blob Storage.

airflow.providers.microsoft.azure.sensors.wasb.WasbBlobSensor

View Docs Source

S

WasbPrefixSensor

Wait for blobs matching a prefix to arrive on Azure Blob Storage.

airflow.providers.microsoft.azure.sensors.wasb.WasbPrefixSensor

View Docs Source

S

AzureDataFactoryPipelineRunStatusSensor

Checks the status of a pipeline run.

airflow.providers.microsoft.azure.sensors.data_factory.AzureDataFactoryPipelineRunStatusSensor

View Docs Source

S

MSGraphSensor

A Microsoft Graph API sensor which allows you to poll an async REST call to the Microsoft Graph API.

airflow.providers.microsoft.azure.sensors.msgraph.MSGraphSensor

View Docs Source

T

ADFPipelineRunStatusSensorTrigger

Trigger with params to run the task when the ADF Pipeline is running.

airflow.providers.microsoft.azure.triggers.data_factory.ADFPipelineRunStatusSensorTrigger

View Docs Source

T

AzureDataFactoryTrigger

Trigger when the Azure data factory pipeline job finishes.

airflow.providers.microsoft.azure.triggers.data_factory.AzureDataFactoryTrigger

View Docs Source

T

WasbBlobSensorTrigger

Check for existence of the given blob in the provided container.

airflow.providers.microsoft.azure.triggers.wasb.WasbBlobSensorTrigger

View Docs Source

T

WasbPrefixSensorTrigger

Check for the existence of a blob with the given prefix in the provided container.

airflow.providers.microsoft.azure.triggers.wasb.WasbPrefixSensorTrigger

View Docs Source

T

MSGraphTrigger

A Microsoft Graph API trigger which allows you to execute an async REST call to the Microsoft Graph API.

airflow.providers.microsoft.azure.triggers.msgraph.MSGraphTrigger

View Docs Source

T

PowerBIDatasetListTrigger

Triggers a call to the API to request the available dataset IDs.

airflow.providers.microsoft.azure.triggers.powerbi.PowerBIDatasetListTrigger

View Docs Source

T

PowerBITrigger

Triggers when Power BI dataset refresh is completed.

airflow.providers.microsoft.azure.triggers.powerbi.PowerBITrigger

View Docs Source

T

PowerBIWorkspaceListTrigger

Triggers a call to the API to request the available workspace IDs.

airflow.providers.microsoft.azure.triggers.powerbi.PowerBIWorkspaceListTrigger

View Docs Source

T

AzureServiceBusQueueTrigger

Trigger for Azure Service Bus Queue message processing.

airflow.providers.microsoft.azure.triggers.message_bus.AzureServiceBusQueueTrigger

View Docs Source

T

AzureServiceBusSubscriptionTrigger

Trigger for Azure Service Bus Topic Subscription message processing.

airflow.providers.microsoft.azure.triggers.message_bus.AzureServiceBusSubscriptionTrigger

View Docs Source

T

LocalFilesystemToADLSOperator

Upload file(s) to Azure Data Lake.

airflow.providers.microsoft.azure.transfers.local_to_adls.LocalFilesystemToADLSOperator

View Docs Source

T

OracleToAzureDataLakeOperator

Runs the query against Oracle and stores the file locally before loading it into Azure Data Lake.

airflow.providers.microsoft.azure.transfers.oracle_to_azure_data_lake.OracleToAzureDataLakeOperator

View Docs Source

T

LocalFilesystemToWasbOperator

Uploads a file to Azure Blob Storage.

airflow.providers.microsoft.azure.transfers.local_to_wasb.LocalFilesystemToWasbOperator

View Docs Source

T

SFTPToWasbOperator

Transfer files to Azure Blob Storage from SFTP server.

airflow.providers.microsoft.azure.transfers.sftp_to_wasb.SFTPToWasbOperator

View Docs Source

T

S3ToAzureBlobStorageOperator

Operator to move data from and AWS S3 Bucket to Microsoft Azure Blob Storage.

airflow.providers.microsoft.azure.transfers.s3_to_wasb.S3ToAzureBlobStorageOperator

View Docs Source

S

AzureKeyVaultBackend

Retrieves Airflow Connections or Variables from Azure Key Vault secrets.

airflow.providers.microsoft.azure.secrets.key_vault.AzureKeyVaultBackend

View Docs Source

L

WasbTaskHandler

WasbTaskHandler is a python log handler that handles and reads task instance logs.

airflow.providers.microsoft.azure.log.wasb_task_handler.WasbTaskHandler

View Docs Source