`airflow.providers.microsoft.azure.hooks.data_factory`¶

Module Contents¶

Classes¶

`PipelineRunInfo`	Type class for the pipeline run info dictionary.
`AzureDataFactoryPipelineRunStatus`	Azure Data Factory pipeline operation statuses.
`AzureDataFactoryHook`	A hook to interact with Azure Data Factory.

Functions¶

provide_targeted_factory(func: Callable) → Callable

Provide the targeted factory to the decorated function in case it isn't specified.

Attributes¶

Credentials

airflow.providers.microsoft.azure.hooks.data_factory.Credentials[source]¶

airflow.providers.microsoft.azure.hooks.data_factory.provide_targeted_factory(func: Callable) → Callable[source]¶

Provide the targeted factory to the decorated function in case it isn't specified.

If resource_group_name or factory_name is not provided it defaults to the value specified in the connection extras.

class airflow.providers.microsoft.azure.hooks.data_factory.PipelineRunInfo[source]¶

Bases: airflow.typing_compat.TypedDict

Type class for the pipeline run info dictionary.

run_id :str[source]¶

factory_name :Optional[str][source]¶

resource_group_name :Optional[str][source]¶

class airflow.providers.microsoft.azure.hooks.data_factory.AzureDataFactoryPipelineRunStatus[source]¶

Azure Data Factory pipeline operation statuses.

QUEUED = Queued[source]¶

IN_PROGRESS = InProgress[source]¶

SUCCEEDED = Succeeded[source]¶

FAILED = Failed[source]¶

CANCELING = Canceling[source]¶

CANCELLED = Cancelled[source]¶

TERMINAL_STATUSES[source]¶

exception airflow.providers.microsoft.azure.hooks.data_factory.AzureDataFactoryPipelineRunException[source]¶

Bases: airflow.exceptions.AirflowException

An exception that indicates a pipeline run failed to complete.

class airflow.providers.microsoft.azure.hooks.data_factory.AzureDataFactoryHook(azure_data_factory_conn_id: str = default_conn_name)[source]¶

Bases: airflow.hooks.base.BaseHook

A hook to interact with Azure Data Factory.

Parameters: azure_data_factory_conn_id (str) -- The Azure Data Factory connection id.

conn_type :str = azure_data_factory[source]¶

conn_name_attr :str = azure_data_factory_conn_id[source]¶

default_conn_name :str = azure_data_factory_default[source]¶

hook_name :str = Azure Data Factory[source]¶

static get_connection_form_widgets() → Dict[str, Any][source]¶: Returns connection widgets to add to connection form

static get_ui_field_behaviour() → Dict[source]¶: Returns custom field behaviour

get_conn(self) → azure.mgmt.datafactory.DataFactoryManagementClient[source]¶: Returns connection for the hook.

get_factory(self, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.Factory[source]¶

Get the factory.

Parameters

resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Returns

The factory.

update_factory(self, factory: azure.mgmt.datafactory.models.Factory, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.Factory[source]¶

Update the factory.

Parameters

factory -- The factory resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the factory does not exist.

Returns

The factory.

create_factory(self, factory: azure.mgmt.datafactory.models.Factory, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.Factory[source]¶

Create the factory.

Parameters

factory -- The factory resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the factory already exists.

Returns

The factory.

delete_factory(self, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → None[source]¶

Delete the factory.

Parameters

resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

get_linked_service(self, linked_service_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.LinkedServiceResource[source]¶

Get the linked service.

Parameters

linked_service_name -- The linked service name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Returns

The linked service.

update_linked_service(self, linked_service_name: str, linked_service: azure.mgmt.datafactory.models.LinkedServiceResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.LinkedServiceResource[source]¶

Update the linked service.

Parameters

linked_service_name -- The linked service name.
linked_service -- The linked service resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the linked service does not exist.

Returns

The linked service.

create_linked_service(self, linked_service_name: str, linked_service: azure.mgmt.datafactory.models.LinkedServiceResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.LinkedServiceResource[source]¶

Create the linked service.

Parameters

linked_service_name -- The linked service name.
linked_service -- The linked service resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the linked service already exists.

Returns

The linked service.

delete_linked_service(self, linked_service_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → None[source]¶

Delete the linked service.

Parameters

linked_service_name -- The linked service name.
resource_group_name -- The linked service name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

get_dataset(self, dataset_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.DatasetResource[source]¶

Get the dataset.

Parameters

dataset_name -- The dataset name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Returns

The dataset.

update_dataset(self, dataset_name: str, dataset: azure.mgmt.datafactory.models.DatasetResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.DatasetResource[source]¶

Update the dataset.

Parameters

dataset_name -- The dataset name.
dataset -- The dataset resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the dataset does not exist.

Returns

The dataset.

create_dataset(self, dataset_name: str, dataset: azure.mgmt.datafactory.models.DatasetResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.DatasetResource[source]¶

Create the dataset.

Parameters

dataset_name -- The dataset name.
dataset -- The dataset resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the dataset already exists.

Returns

The dataset.

delete_dataset(self, dataset_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → None[source]¶

Delete the dataset.

Parameters

dataset_name -- The dataset name.
resource_group_name -- The dataset name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

get_pipeline(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.PipelineResource[source]¶

Get the pipeline.

Parameters

pipeline_name -- The pipeline name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Returns

The pipeline.

update_pipeline(self, pipeline_name: str, pipeline: azure.mgmt.datafactory.models.PipelineResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.PipelineResource[source]¶

Update the pipeline.

Parameters

pipeline_name -- The pipeline name.
pipeline -- The pipeline resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the pipeline does not exist.

Returns

The pipeline.

create_pipeline(self, pipeline_name: str, pipeline: azure.mgmt.datafactory.models.PipelineResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.PipelineResource[source]¶

Create the pipeline.

Parameters

pipeline_name -- The pipeline name.
pipeline -- The pipeline resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the pipeline already exists.

Returns

The pipeline.

delete_pipeline(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → None[source]¶

Delete the pipeline.

Parameters

pipeline_name -- The pipeline name.
resource_group_name -- The pipeline name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

run_pipeline(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.CreateRunResponse[source]¶

Run a pipeline.

Parameters

pipeline_name -- The pipeline name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Returns

The pipeline run.

get_pipeline_run(self, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.PipelineRun[source]¶

Get the pipeline run.

Parameters

run_id -- The pipeline run identifier.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Returns

The pipeline run.

get_pipeline_run_status(self, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None) → str[source]¶

Get a pipeline run's current status.

Parameters

run_id -- The pipeline run identifier.
resource_group_name -- The resource group name.
factory_name -- The factory name.

Returns

The status of the pipeline run.

wait_for_pipeline_run_status(self, run_id: str, expected_statuses: Union[str, Set[str]], resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, check_interval: int = 60, timeout: int = 60 * 60 * 24 * 7) → bool[source]¶

Waits for a pipeline run to match an expected status.

Parameters

run_id -- The pipeline run identifier.
expected_statuses -- The desired status(es) to check against a pipeline run's current status.
resource_group_name -- The resource group name.
factory_name -- The factory name.
check_interval -- Time in seconds to check on a pipeline run's status.
timeout -- Time in seconds to wait for a pipeline to reach a terminal status or the expected status.

Returns

Boolean indicating if the pipeline run has reached the expected_status.

cancel_pipeline_run(self, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → None[source]¶

Cancel the pipeline run.

Parameters

run_id -- The pipeline run identifier.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

get_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.TriggerResource[source]¶

Get the trigger.

Parameters

trigger_name -- The trigger name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Returns

The trigger.

update_trigger(self, trigger_name: str, trigger: azure.mgmt.datafactory.models.TriggerResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.TriggerResource[source]¶

Update the trigger.

Parameters

trigger_name -- The trigger name.
trigger -- The trigger resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the trigger does not exist.

Returns

The trigger.

create_trigger(self, trigger_name: str, trigger: azure.mgmt.datafactory.models.TriggerResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.mgmt.datafactory.models.TriggerResource[source]¶

Create the trigger.

Parameters

trigger_name -- The trigger name.
trigger -- The trigger resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the trigger already exists.

Returns

The trigger.

delete_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → None[source]¶

Delete the trigger.

Parameters

trigger_name -- The trigger name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

start_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.core.polling.LROPoller[source]¶

Start the trigger.

Parameters

trigger_name -- The trigger name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Returns

An Azure operation poller.

stop_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → azure.core.polling.LROPoller[source]¶

Stop the trigger.

Parameters

trigger_name -- The trigger name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

Returns

An Azure operation poller.

rerun_trigger(self, trigger_name: str, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → None[source]¶

Rerun the trigger.

Parameters

trigger_name -- The trigger name.
run_id -- The trigger run identifier.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

cancel_trigger(self, trigger_name: str, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config: Any) → None[source]¶

Cancel the trigger.

Parameters

trigger_name -- The trigger name.
run_id -- The trigger run identifier.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.

airflow.providers.microsoft.azure.hooks.data_factory¶

Module Contents¶

Classes¶

Functions¶

Attributes¶

`airflow.providers.microsoft.azure.hooks.data_factory`¶