airflow.providers.microsoft.azure.operators.container_instances¶
Module Contents¶
Classes¶
Start a container on Azure Container Instances. |
Attributes¶
- airflow.providers.microsoft.azure.operators.container_instances.DEFAULT_ENVIRONMENT_VARIABLES: dict[str, str][source]¶
- airflow.providers.microsoft.azure.operators.container_instances.DEFAULT_SECURED_VARIABLES: Sequence[str] = [][source]¶
- airflow.providers.microsoft.azure.operators.container_instances.DEFAULT_VOLUMES: Sequence[Volume] = [][source]¶
- class airflow.providers.microsoft.azure.operators.container_instances.AzureContainerInstancesOperator(*, ci_conn_id, resource_group, name, image, region, registry_conn_id=None, environment_variables=None, secured_variables=None, volumes=None, memory_in_gb=None, cpu=None, gpu=None, command=None, remove_on_error=True, fail_if_exists=True, tags=None, xcom_all=None, os_type='Linux', restart_policy='Never', ip_address=None, ports=None, subnet_ids=None, dns_config=None, diagnostics=None, priority='Regular', **kwargs)[source]¶
Bases:
airflow.models.BaseOperatorStart a container on Azure Container Instances.
- Parameters
ci_conn_id (str) – connection id of a service principal which will be used to start the container instance
registry_conn_id (str | None) – connection id of a user which can login to a private docker registry. For Azure use Azure connection id
resource_group (str) – name of the resource group wherein this container instance should be started
name (str) – name of this container instance. Please note this name has to be unique in order to run containers in parallel.
image (str) – the docker image to be used
region (str) – the region wherein this container instance should be started
environment_variables (dict | None) – key,value pairs containing environment variables which will be passed to the running container
secured_variables (str | None) – names of environmental variables that should not be exposed outside the container (typically passwords).
volumes (list | None) – list of
Volumetuples to be mounted to the container. Currently only Azure Fileshares are supported.memory_in_gb (Any | None) – the amount of memory to allocate to this container
cpu (Any | None) – the number of cpus to allocate to this container
gpu (Any | None) – GPU Resource for the container.
command (list[str] | None) – the command to run inside the container
container_timeout – max time allowed for the execution of the container instance.
tags (dict[str, str] | None) – azure tags as dict of str:str
xcom_all (bool | None) – Control if logs are pushed to XCOM similarly to how DockerOperator does. Possible values include: ‘None’, ‘True’, ‘False’. Defaults to ‘None’, meaning no logs are pushed to XCOM which is the historical behaviour. ‘True’ means push all logs to XCOM which may run the risk of hitting XCOM size limits. ‘False’ means push only the last line of the logs to XCOM. However, the logs are pushed into XCOM under “logs”, not return_value to avoid breaking the existing behaviour.
os_type (str) – The operating system type required by the containers in the container group. Possible values include: ‘Windows’, ‘Linux’
restart_policy (str) – Restart policy for all containers within the container group. Possible values include: ‘Always’, ‘OnFailure’, ‘Never’
ip_address (azure.mgmt.containerinstance.models.IpAddress | None) – The IP address type of the container group.
subnet_ids (list[azure.mgmt.containerinstance.models.ContainerGroupSubnetId] | None) – The subnet resource IDs for a container group
dns_config (azure.mgmt.containerinstance.models.DnsConfiguration | None) – The DNS configuration for a container group.
diagnostics (azure.mgmt.containerinstance.models.ContainerGroupDiagnostics | None) – Container group diagnostic information (Log Analytics).
priority (str | None) – Container group priority, Possible values include: ‘Regular’, ‘Spot’
Example:
AzureContainerInstancesOperator( ci_conn_id="azure_service_principal", registry_conn_id="azure_registry_user", resource_group="my-resource-group", name="my-container-name-{{ ds }}", image="myprivateregistry.azurecr.io/my_container:latest", region="westeurope", environment_variables={ "MODEL_PATH": "my_value", "POSTGRES_LOGIN": "{{ macros.connection('postgres_default').login }}", "POSTGRES_PASSWORD": "{{ macros.connection('postgres_default').password }}", "JOB_GUID": "{{ ti.xcom_pull(task_ids='task1', key='guid') }}", }, secured_variables=["POSTGRES_PASSWORD"], volumes=[ ("azure_container_instance_conn_id", "my_storage_container", "my_fileshare", "/input-data", True), ], memory_in_gb=14.0, cpu=4.0, gpu=GpuResource(count=1, sku="K80"), subnet_ids=[ { "id": "/subscriptions/00000000-0000-0000-0000-00000000000/resourceGroups/my_rg/providers/Microsoft.Network/virtualNetworks/my_vnet/subnets/my_subnet" } ], dns_config={"name_servers": ["10.0.0.10", "10.0.0.11"]}, diagnostics={ "log_analytics": { "workspaceId": "workspaceid", "workspaceKey": "workspaceKey", } }, priority="Regular", command=["/bin/echo", "world"], task_id="start_container", )
- template_fields: Sequence[str] = ('name', 'image', 'command', 'environment_variables', 'volumes')[source]¶