airflow.providers.microsoft.azure.hooks.azure_batch

Module Contents

class airflow.providers.microsoft.azure.hooks.azure_batch.AzureBatchHook(azure_batch_conn_id: str = default_conn_name)[source]

Bases: airflow.hooks.base.BaseHook

Hook for Azure Batch APIs

Account name and account key should be in login and password parameters. The account url should be in extra parameter as account_url

conn_name_attr = azure_batch_conn_id[source]
default_conn_name = azure_batch_default[source]
conn_type = azure_batch[source]
hook_name = Azure Batch Service[source]
_connection(self)[source]

Get connected to azure batch service

get_conn(self)[source]

Get the batch client connection

Returns

Azure batch client

configure_pool(self, pool_id: str, vm_size: Optional[str] = None, vm_publisher: Optional[str] = None, vm_offer: Optional[str] = None, sku_starts_with: Optional[str] = None, vm_sku: Optional[str] = None, vm_version: Optional[str] = None, vm_node_agent_sku_id: Optional[str] = None, os_family: Optional[str] = None, os_version: Optional[str] = None, display_name: Optional[str] = None, target_dedicated_nodes: Optional[int] = None, use_latest_image_and_sku: bool = False, **kwargs)[source]

Configures a pool

Parameters

  • pool_id (str) -- A string that uniquely identifies the Pool within the Account

  • vm_size (str) -- The size of virtual machines in the Pool.

  • display_name (str) -- The display name for the Pool

  • target_dedicated_nodes (Optional[int]) -- The desired number of dedicated Compute Nodes in the Pool.

  • use_latest_image_and_sku (bool) -- Whether to use the latest verified vm image and sku

  • vm_publisher (Optional[str]) -- The publisher of the Azure Virtual Machines Marketplace Image. For example, Canonical or MicrosoftWindowsServer.

  • vm_offer (Optional[str]) -- The offer type of the Azure Virtual Machines Marketplace Image. For example, UbuntuServer or WindowsServer.

  • sku_starts_with (Optional[str]) -- The start name of the sku to search

  • vm_sku (Optional[str]) -- The name of the virtual machine sku to use

  • vm_version -- The version of the virtual machine

  • vm_version -- str

  • vm_node_agent_sku_id (Optional[str]) -- The node agent sku id of the virtual machine

  • os_family (Optional[str]) -- The Azure Guest OS family to be installed on the virtual machines in the Pool.

  • os_version (Optional[str]) -- The OS family version

create_pool(self, pool: PoolAddParameter)[source]

Creates a pool if not already existing

Parameters

pool (batch_models.PoolAddParameter) -- the pool object to create

_get_latest_verified_image_vm_and_sku(self, publisher: Optional[str] = None, offer: Optional[str] = None, sku_starts_with: Optional[str] = None)[source]

Get latest verified image vm and sku

Parameters

  • publisher (str) -- The publisher of the Azure Virtual Machines Marketplace Image. For example, Canonical or MicrosoftWindowsServer.

  • offer (str) -- The offer type of the Azure Virtual Machines Marketplace Image. For example, UbuntuServer or WindowsServer.

  • sku_starts_with (str) -- The start name of the sku to search

wait_for_all_node_state(self, pool_id: str, node_state: Set)[source]

Wait for all nodes in a pool to reach given states

Parameters

  • pool_id (str) -- A string that identifies the pool

  • node_state (set) -- A set of batch_models.ComputeNodeState

configure_job(self, job_id: str, pool_id: str, display_name: Optional[str] = None, **kwargs)[source]

Configures a job for use in the pool

Parameters

  • job_id (str) -- A string that uniquely identifies the job within the account

  • pool_id (str) -- A string that identifies the pool

  • display_name (str) -- The display name for the job

create_job(self, job: JobAddParameter)[source]

Creates a job in the pool

Parameters

job (batch_models.JobAddParameter) -- The job object to create

configure_task(self, task_id: str, command_line: str, display_name: Optional[str] = None, container_settings=None, **kwargs)[source]

Creates a task

Parameters

  • task_id (str) -- A string that identifies the task to create

  • command_line (str) -- The command line of the Task.

  • display_name (str) -- A display name for the Task

  • container_settings (batch_models.TaskContainerSettings) -- The settings for the container under which the Task runs. If the Pool that will run this Task has containerConfiguration set, this must be set as well. If the Pool that will run this Task doesn't have containerConfiguration set, this must not be set.

add_single_task_to_job(self, job_id: str, task: TaskAddParameter)[source]

Add a single task to given job if it doesn't exist

Parameters

  • job_id (str) -- A string that identifies the given job

  • task (batch_models.TaskAddParameter) -- The task to add

wait_for_job_tasks_to_complete(self, job_id: str, timeout: int)[source]

Wait for tasks in a particular job to complete

Parameters

  • job_id (str) -- A string that identifies the job

  • timeout (int) -- The amount of time to wait before timing out in minutes

Was this entry helpful?