airflow.providers.microsoft.azure.hooks.azure_data_factory
¶
Module Contents¶
-
airflow.providers.microsoft.azure.hooks.azure_data_factory.
provide_targeted_factory
(func: Callable) → Callable[source]¶ -
Provide the targeted factory to the decorated function in case it isn't specified.
If
resource_group_name
orfactory_name
is not provided it defaults to the value specified in the connection extras.
-
class
airflow.providers.microsoft.azure.hooks.azure_data_factory.
AzureDataFactoryHook
(conn_id: str = 'azure_data_factory_default')[source]¶ Bases:
airflow.hooks.base.BaseHook
A hook to interact with Azure Data Factory.
- Parameters
conn_id -- The Azure Data Factory connection id.
-
get_factory
(self, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Get the factory.
- Parameters
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Returns
The factory.
-
_factory_exists
(self, resource_group_name, factory_name)[source]¶ Return whether or not the factory already exists.
-
update_factory
(self, factory: Factory, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Update the factory.
- Parameters
factory -- The factory resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Raises
AirflowException -- If the factory does not exist.
- Returns
The factory.
-
create_factory
(self, factory: Factory, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Create the factory.
- Parameters
factory -- The factory resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Raises
AirflowException -- If the factory already exists.
- Returns
The factory.
-
delete_factory
(self, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Delete the factory.
- Parameters
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
-
get_linked_service
(self, linked_service_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Get the linked service.
- Parameters
linked_service_name -- The linked service name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Returns
The linked service.
-
_linked_service_exists
(self, resource_group_name, factory_name, linked_service_name)[source]¶ Return whether or not the linked service already exists.
-
update_linked_service
(self, linked_service_name: str, linked_service: LinkedService, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Update the linked service.
- Parameters
linked_service_name -- The linked service name.
linked_service -- The linked service resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Raises
AirflowException -- If the linked service does not exist.
- Returns
The linked service.
-
create_linked_service
(self, linked_service_name: str, linked_service: LinkedService, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Create the linked service.
- Parameters
linked_service_name -- The linked service name.
linked_service -- The linked service resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Raises
AirflowException -- If the linked service already exists.
- Returns
The linked service.
-
delete_linked_service
(self, linked_service_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Delete the linked service.
- Parameters
linked_service_name -- The linked service name.
resource_group_name -- The linked service name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
-
get_dataset
(self, dataset_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Get the dataset.
- Parameters
dataset_name -- The dataset name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Returns
The dataset.
-
_dataset_exists
(self, resource_group_name, factory_name, dataset_name)[source]¶ Return whether or not the dataset already exists.
-
update_dataset
(self, dataset_name: str, dataset: Dataset, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Update the dataset.
- Parameters
dataset_name -- The dataset name.
dataset -- The dataset resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Raises
AirflowException -- If the dataset does not exist.
- Returns
The dataset.
-
create_dataset
(self, dataset_name: str, dataset: Dataset, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Create the dataset.
- Parameters
dataset_name -- The dataset name.
dataset -- The dataset resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Raises
AirflowException -- If the dataset already exists.
- Returns
The dataset.
-
delete_dataset
(self, dataset_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Delete the dataset.
- Parameters
dataset_name -- The dataset name.
resource_group_name -- The dataset name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
-
get_pipeline
(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Get the pipeline.
- Parameters
pipeline_name -- The pipeline name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Returns
The pipeline.
-
_pipeline_exists
(self, resource_group_name, factory_name, pipeline_name)[source]¶ Return whether or not the pipeline already exists.
-
update_pipeline
(self, pipeline_name: str, pipeline: PipelineResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Update the pipeline.
- Parameters
pipeline_name -- The pipeline name.
pipeline -- The pipeline resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Raises
AirflowException -- If the pipeline does not exist.
- Returns
The pipeline.
-
create_pipeline
(self, pipeline_name: str, pipeline: PipelineResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Create the pipeline.
- Parameters
pipeline_name -- The pipeline name.
pipeline -- The pipeline resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Raises
AirflowException -- If the pipeline already exists.
- Returns
The pipeline.
-
delete_pipeline
(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Delete the pipeline.
- Parameters
pipeline_name -- The pipeline name.
resource_group_name -- The pipeline name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
-
run_pipeline
(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Run a pipeline.
- Parameters
pipeline_name -- The pipeline name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Returns
The pipeline run.
-
get_pipeline_run
(self, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Get the pipeline run.
- Parameters
run_id -- The pipeline run identifier.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Returns
The pipeline run.
-
cancel_pipeline_run
(self, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Cancel the pipeline run.
- Parameters
run_id -- The pipeline run identifier.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
-
get_trigger
(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Get the trigger.
- Parameters
trigger_name -- The trigger name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Returns
The trigger.
-
_trigger_exists
(self, resource_group_name, factory_name, trigger_name)[source]¶ Return whether or not the trigger already exists.
-
update_trigger
(self, trigger_name: str, trigger: Trigger, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Update the trigger.
- Parameters
trigger_name -- The trigger name.
trigger -- The trigger resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Raises
AirflowException -- If the trigger does not exist.
- Returns
The trigger.
-
create_trigger
(self, trigger_name: str, trigger: Trigger, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Create the trigger.
- Parameters
trigger_name -- The trigger name.
trigger -- The trigger resource definition.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Raises
AirflowException -- If the trigger already exists.
- Returns
The trigger.
-
delete_trigger
(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Delete the trigger.
- Parameters
trigger_name -- The trigger name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
-
start_trigger
(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Start the trigger.
- Parameters
trigger_name -- The trigger name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Returns
An Azure operation poller.
-
stop_trigger
(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Stop the trigger.
- Parameters
trigger_name -- The trigger name.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
- Returns
An Azure operation poller.
-
rerun_trigger
(self, trigger_name: str, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Rerun the trigger.
- Parameters
trigger_name -- The trigger name.
run_id -- The trigger run identifier.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.
-
cancel_trigger
(self, trigger_name: str, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)[source]¶ Cancel the trigger.
- Parameters
trigger_name -- The trigger name.
run_id -- The trigger run identifier.
resource_group_name -- The resource group name.
factory_name -- The factory name.
config -- Extra parameters for the ADF client.