airflow.providers.microsoft.azure.hooks.batch

Module Contents

Classes

AzureBatchHook

Hook for Azure Batch APIs
class airflow.providers.microsoft.azure.hooks.batch.AzureBatchHook(azure_batch_conn_id=default_conn_name)[source]

Bases: airflow.hooks.base.BaseHook

Hook for Azure Batch APIs

Parameters

azure_batch_conn_id (str) – Azure Batch connection id of a service principal which will be used to start the container instance.

conn_name_attr = 'azure_batch_conn_id'[source]
default_conn_name = 'azure_batch_default'[source]
conn_type = 'azure_batch'[source]
hook_name = 'Azure Batch Service'[source]
static get_connection_form_widgets()[source]

Returns connection widgets to add to connection form

static get_ui_field_behaviour()[source]

Returns custom field behaviour

get_conn()[source]

Get the Batch client connection

Returns

Azure Batch client

configure_pool(pool_id, vm_size, vm_node_agent_sku_id, vm_publisher=None, vm_offer=None, sku_starts_with=None, vm_sku=None, vm_version=None, os_family=None, os_version=None, display_name=None, target_dedicated_nodes=None, use_latest_image_and_sku=False, **kwargs)[source]

Configures a pool

Parameters

  • pool_id (str) – A string that uniquely identifies the Pool within the Account

  • vm_size (str) – The size of virtual machines in the Pool.

  • display_name (str | None) – The display name for the Pool

  • target_dedicated_nodes (int | None) – The desired number of dedicated Compute Nodes in the Pool.

  • use_latest_image_and_sku (bool) – Whether to use the latest verified vm image and sku

  • vm_publisher (str | None) – The publisher of the Azure Virtual Machines Marketplace Image. For example, Canonical or MicrosoftWindowsServer.

  • vm_offer (str | None) – The offer type of the Azure Virtual Machines Marketplace Image. For example, UbuntuServer or WindowsServer.

  • sku_starts_with (str | None) – The start name of the sku to search

  • vm_sku (str | None) – The name of the virtual machine sku to use

  • vm_version (str | None) – The version of the virtual machine

  • vm_version – str

  • vm_node_agent_sku_id (str) – The node agent sku id of the virtual machine

  • os_family (str | None) – The Azure Guest OS family to be installed on the virtual machines in the Pool.

  • os_version (str | None) – The OS family version

create_pool(pool)[source]

Creates a pool if not already existing

Parameters

pool (azure.batch.models.PoolAddParameter) – the pool object to create

wait_for_all_node_state(pool_id, node_state)[source]

Wait for all nodes in a pool to reach given states

Parameters

  • pool_id (str) – A string that identifies the pool

  • node_state (set) – A set of batch_models.ComputeNodeState

configure_job(job_id, pool_id, display_name=None, **kwargs)[source]

Configures a job for use in the pool

Parameters

  • job_id (str) – A string that uniquely identifies the job within the account

  • pool_id (str) – A string that identifies the pool

  • display_name (str | None) – The display name for the job

create_job(job)[source]

Creates a job in the pool

Parameters

job (azure.batch.models.JobAddParameter) – The job object to create

configure_task(task_id, command_line, display_name=None, container_settings=None, **kwargs)[source]

Creates a task

Parameters

  • task_id (str) – A string that identifies the task to create

  • command_line (str) – The command line of the Task.

  • display_name (str | None) – A display name for the Task

  • container_settings – The settings for the container under which the Task runs. If the Pool that will run this Task has containerConfiguration set, this must be set as well. If the Pool that will run this Task doesn’t have containerConfiguration set, this must not be set.

add_single_task_to_job(job_id, task)[source]

Add a single task to given job if it doesn’t exist

Parameters

  • job_id (str) – A string that identifies the given job

  • task (azure.batch.models.TaskAddParameter) – The task to add

wait_for_job_tasks_to_complete(job_id, timeout)[source]

Wait for tasks in a particular job to complete

Parameters

  • job_id (str) – A string that identifies the job

  • timeout (int) – The amount of time to wait before timing out in minutes

test_connection()[source]

Test a configured Azure Batch connection.

Was this entry helpful?