airflow.contrib.operators.azure_container_instances_operator

Module Contents

airflow.contrib.operators.azure_container_instances_operator.Volume[source]
airflow.contrib.operators.azure_container_instances_operator.DEFAULT_ENVIRONMENT_VARIABLES[source]
airflow.contrib.operators.azure_container_instances_operator.DEFAULT_VOLUMES = [][source]
airflow.contrib.operators.azure_container_instances_operator.DEFAULT_MEMORY_IN_GB = 2.0[source]
airflow.contrib.operators.azure_container_instances_operator.DEFAULT_CPU = 1.0[source]
class airflow.contrib.operators.azure_container_instances_operator.AzureContainerInstancesOperator(ci_conn_id, registry_conn_id, resource_group, name, image, region, environment_variables=None, volumes=None, memory_in_gb=None, cpu=None, command=None, remove_on_error=True, fail_if_exists=True, *args, **kwargs)[source]

Bases:airflow.models.BaseOperator

Start a container on Azure Container Instances

Parameters

  • ci_conn_id (str) – connection id of a service principal which will be used to start the container instance

  • registry_conn_id (str) – connection id of a user which can login to a private docker registry. If None, we assume a public registry

  • resource_group (str) – name of the resource group wherein this container instance should be started

  • name (str) – name of this container instance. Please note this name has to be unique in order to run containers in parallel.

  • image (str) – the docker image to be used

  • region (str) – the region wherein this container instance should be started

  • environment_variables (dict) – key,value pairs containing environment variables which will be passed to the running container

  • volumes (list[<conn_id, account_name, share_name, mount_path, read_only>]) – list of volumes to be mounted to the container. Currently only Azure Fileshares are supported.

  • memory_in_gb (double) – the amount of memory to allocate to this container

  • cpu (double) – the number of cpus to allocate to this container

  • command (str) – the command to run inside the container

Example

>>>  a = AzureContainerInstancesOperator(
            'azure_service_principal',
            'azure_registry_user',
            'my-resource-group',
            'my-container-name-{{ ds }}',
            'myprivateregistry.azurecr.io/my_container:latest',
            'westeurope',
            {'EXECUTION_DATE': '{{ ds }}'},
            [('azure_wasb_conn_id',
              'my_storage_container',
              'my_fileshare',
              '/input-data',
              True),],
            memory_in_gb=14.0,
            cpu=4.0,
            command='python /app/myfile.py',
            task_id='start_container'
        )
template_fields = ['name', 'environment_variables'][source]
execute(self, context)[source]
_monitor_logging(self, ci_hook, resource_group, name)[source]
_log_last(self, logs, last_line_logged)[source]