airflow.providers.microsoft.azure.hooks.azure_data_factory

Module Contents

airflow.providers.microsoft.azure.hooks.azure_data_factory.provide_targeted_factory(func: Callable) → Callable[source]
Provide the targeted factory to the decorated function in case it isn't specified.

If resource_group_name or factory_name is not provided it defaults to the value specified in the connection extras.

class airflow.providers.microsoft.azure.hooks.azure_data_factory.AzureDataFactoryHook(conn_id: str = 'azure_data_factory_default')[source]

Bases: airflow.hooks.base.BaseHook

A hook to interact with Azure Data Factory.

Parameters

conn_id -- The Azure Data Factory connection id.

get_conn(self)[source]
get_factory(self, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Get the factory.

Parameters
  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Returns

The factory.

_factory_exists(self, resource_group_name, factory_name)[source]

Return whether or not the factory already exists.

update_factory(self, factory: Factory, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Update the factory.

Parameters
  • factory -- The factory resource definition.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the factory does not exist.

Returns

The factory.

create_factory(self, factory: Factory, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Create the factory.

Parameters
  • factory -- The factory resource definition.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the factory already exists.

Returns

The factory.

delete_factory(self, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Delete the factory.

Parameters
  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

get_linked_service(self, linked_service_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Get the linked service.

Parameters
  • linked_service_name -- The linked service name.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Returns

The linked service.

_linked_service_exists(self, resource_group_name, factory_name, linked_service_name)[source]

Return whether or not the linked service already exists.

update_linked_service(self, linked_service_name: str, linked_service: LinkedService, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Update the linked service.

Parameters
  • linked_service_name -- The linked service name.

  • linked_service -- The linked service resource definition.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the linked service does not exist.

Returns

The linked service.

create_linked_service(self, linked_service_name: str, linked_service: LinkedService, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Create the linked service.

Parameters
  • linked_service_name -- The linked service name.

  • linked_service -- The linked service resource definition.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the linked service already exists.

Returns

The linked service.

delete_linked_service(self, linked_service_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Delete the linked service.

Parameters
  • linked_service_name -- The linked service name.

  • resource_group_name -- The linked service name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

get_dataset(self, dataset_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Get the dataset.

Parameters
  • dataset_name -- The dataset name.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Returns

The dataset.

_dataset_exists(self, resource_group_name, factory_name, dataset_name)[source]

Return whether or not the dataset already exists.

update_dataset(self, dataset_name: str, dataset: Dataset, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Update the dataset.

Parameters
  • dataset_name -- The dataset name.

  • dataset -- The dataset resource definition.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the dataset does not exist.

Returns

The dataset.

create_dataset(self, dataset_name: str, dataset: Dataset, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Create the dataset.

Parameters
  • dataset_name -- The dataset name.

  • dataset -- The dataset resource definition.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the dataset already exists.

Returns

The dataset.

delete_dataset(self, dataset_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Delete the dataset.

Parameters
  • dataset_name -- The dataset name.

  • resource_group_name -- The dataset name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

get_pipeline(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Get the pipeline.

Parameters
  • pipeline_name -- The pipeline name.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Returns

The pipeline.

_pipeline_exists(self, resource_group_name, factory_name, pipeline_name)[source]

Return whether or not the pipeline already exists.

update_pipeline(self, pipeline_name: str, pipeline: PipelineResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Update the pipeline.

Parameters
  • pipeline_name -- The pipeline name.

  • pipeline -- The pipeline resource definition.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the pipeline does not exist.

Returns

The pipeline.

create_pipeline(self, pipeline_name: str, pipeline: PipelineResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Create the pipeline.

Parameters
  • pipeline_name -- The pipeline name.

  • pipeline -- The pipeline resource definition.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the pipeline already exists.

Returns

The pipeline.

delete_pipeline(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Delete the pipeline.

Parameters
  • pipeline_name -- The pipeline name.

  • resource_group_name -- The pipeline name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

run_pipeline(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Run a pipeline.

Parameters
  • pipeline_name -- The pipeline name.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Returns

The pipeline run.

get_pipeline_run(self, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Get the pipeline run.

Parameters
  • run_id -- The pipeline run identifier.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Returns

The pipeline run.

cancel_pipeline_run(self, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Cancel the pipeline run.

Parameters
  • run_id -- The pipeline run identifier.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

get_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Get the trigger.

Parameters
  • trigger_name -- The trigger name.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Returns

The trigger.

_trigger_exists(self, resource_group_name, factory_name, trigger_name)[source]

Return whether or not the trigger already exists.

update_trigger(self, trigger_name: str, trigger: Trigger, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Update the trigger.

Parameters
  • trigger_name -- The trigger name.

  • trigger -- The trigger resource definition.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the trigger does not exist.

Returns

The trigger.

create_trigger(self, trigger_name: str, trigger: Trigger, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Create the trigger.

Parameters
  • trigger_name -- The trigger name.

  • trigger -- The trigger resource definition.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Raises

AirflowException -- If the trigger already exists.

Returns

The trigger.

delete_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Delete the trigger.

Parameters
  • trigger_name -- The trigger name.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

start_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Start the trigger.

Parameters
  • trigger_name -- The trigger name.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Returns

An Azure operation poller.

stop_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Stop the trigger.

Parameters
  • trigger_name -- The trigger name.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Returns

An Azure operation poller.

rerun_trigger(self, trigger_name: str, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Rerun the trigger.

Parameters
  • trigger_name -- The trigger name.

  • run_id -- The trigger run identifier.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

cancel_trigger(self, trigger_name: str, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]

Cancel the trigger.

Parameters
  • trigger_name -- The trigger name.

  • run_id -- The trigger run identifier.

  • resource_group_name -- The resource group name.

  • factory_name -- The factory name.

  • config -- Extra parameters for the ADF client.

Was this entry helpful?