airflow.providers.microsoft.azure.operators.azure_container_instances

Module Contents

airflow.providers.microsoft.azure.operators.azure_container_instances.Volume[source]
airflow.providers.microsoft.azure.operators.azure_container_instances.DEFAULT_ENVIRONMENT_VARIABLES :Dict[str, str][source]
airflow.providers.microsoft.azure.operators.azure_container_instances.DEFAULT_SECURED_VARIABLES :Sequence[str] = [][source]
airflow.providers.microsoft.azure.operators.azure_container_instances.DEFAULT_VOLUMES :Sequence[Volume] = [][source]
airflow.providers.microsoft.azure.operators.azure_container_instances.DEFAULT_MEMORY_IN_GB = 2.0[source]
airflow.providers.microsoft.azure.operators.azure_container_instances.DEFAULT_CPU = 1.0[source]
class airflow.providers.microsoft.azure.operators.azure_container_instances.AzureContainerInstancesOperator(*, ci_conn_id: str, registry_conn_id: Optional[str], resource_group: str, name: str, image: str, region: str, environment_variables: Optional[dict] = None, secured_variables: Optional[str] = None, volumes: Optional[list] = None, memory_in_gb: Optional[Any] = None, cpu: Optional[Any] = None, gpu: Optional[Any] = None, command: Optional[List[str]] = None, remove_on_error: bool = True, fail_if_exists: bool = True, tags: Optional[Dict[str, str]] = None, os_type: str = 'Linux', restart_policy: str = 'Never', ip_address: Optional[IpAddress] = None, ports: Optional[List[ContainerPort]] = None, **kwargs)[source]

Bases: airflow.models.BaseOperator

Start a container on Azure Container Instances

Parameters

  • ci_conn_id (str) -- connection id of a service principal which will be used to start the container instance

  • registry_conn_id (Optional[str]) -- connection id of a user which can login to a private docker registry. If None, we assume a public registry

  • resource_group (str) -- name of the resource group wherein this container instance should be started

  • name (str) -- name of this container instance. Please note this name has to be unique in order to run containers in parallel.

  • image (str) -- the docker image to be used

  • region (str) -- the region wherein this container instance should be started

  • environment_variables (Optional[dict]) -- key,value pairs containing environment variables which will be passed to the running container

  • secured_variables (Optional[str]) -- names of environmental variables that should not be exposed outside the container (typically passwords).

  • volumes (list[<conn_id, account_name, share_name, mount_path, read_only>]) -- list of Volume tuples to be mounted to the container. Currently only Azure Fileshares are supported.

  • memory_in_gb (double) -- the amount of memory to allocate to this container

  • cpu (double) -- the number of cpus to allocate to this container

  • gpu (azure.mgmt.containerinstance.models.GpuResource) -- GPU Resource for the container.

  • command (Optional[List[str]]) -- the command to run inside the container

  • container_timeout (datetime.timedelta) -- max time allowed for the execution of the container instance.

  • tags (Optional[dict[str, str]]) -- azure tags as dict of str:str

  • os_type (str) -- The operating system type required by the containers in the container group. Possible values include: 'Windows', 'Linux'

  • restart_policy (str) -- Restart policy for all containers within the container group. Possible values include: 'Always', 'OnFailure', 'Never'

  • ip_address (IpAddress) -- The IP address type of the container group.

Example:

AzureContainerInstancesOperator(
    ci_conn_id = "azure_service_principal",
    registry_conn_id = "azure_registry_user",
    resource_group = "my-resource-group",
    name = "my-container-name-{{ ds }}",
    image = "myprivateregistry.azurecr.io/my_container:latest",
    region = "westeurope",
    environment_variables = {"MODEL_PATH":  "my_value",
     "POSTGRES_LOGIN": "{{ macros.connection('postgres_default').login }}",
     "POSTGRES_PASSWORD": "{{ macros.connection('postgres_default').password }}",
     "JOB_GUID": "{{ ti.xcom_pull(task_ids='task1', key='guid') }}" },
    secured_variables = ['POSTGRES_PASSWORD'],
    volumes = [("azure_wasb_conn_id",
            "my_storage_container",
            "my_fileshare",
            "/input-data",
        True),],
    memory_in_gb=14.0,
    cpu=4.0,
    gpu=GpuResource(count=1, sku='K80'),
    command=["/bin/echo", "world"],
    task_id="start_container"
)
template_fields = ['name', 'image', 'command', 'environment_variables'][source]
execute(self, context: dict)[source]
on_kill(self)[source]
_monitor_logging(self, resource_group: str, name: str)[source]
_log_last(self, logs: Optional[list], last_line_logged: Any)[source]
static _check_name(name: str)[source]

Was this entry helpful?