airflow.providers.amazon.aws.operators.ecs

Module Contents

class airflow.providers.amazon.aws.operators.ecs.ECSProtocol[source]

Bases: airflow.typing_compat.Protocol

A structured Protocol for boto3.client('ecs'). This is used for type hints on ECSOperator.client().

See also

run_task(self, **kwargs)[source]

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html#ECS.Client.run_task

get_waiter(self, x: str)[source]

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html#ECS.Client.get_waiter

describe_tasks(self, cluster: str, tasks)[source]

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html#ECS.Client.describe_tasks

stop_task(self, cluster, task, reason: str)[source]

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html#ECS.Client.stop_task

describe_task_definition(self, taskDefinition: str)[source]

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html#ECS.Client.describe_task_definition

list_tasks(self, cluster: str, launchType: str, desiredStatus: str, family: str)[source]

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html#ECS.Client.list_tasks

class airflow.providers.amazon.aws.operators.ecs.ECSOperator(*, task_definition: str, cluster: str, overrides: dict, aws_conn_id: Optional[str] = None, region_name: Optional[str] = None, launch_type: str = 'EC2', group: Optional[str] = None, placement_constraints: Optional[list] = None, placement_strategy: Optional[list] = None, platform_version: str = 'LATEST', network_configuration: Optional[dict] = None, tags: Optional[dict] = None, awslogs_group: Optional[str] = None, awslogs_region: Optional[str] = None, awslogs_stream_prefix: Optional[str] = None, propagate_tags: Optional[str] = None, reattach: bool = False, **kwargs)[source]

Bases: airflow.models.BaseOperator

Execute a task on AWS ECS (Elastic Container Service)

See also

For more information on how to use this operator, take a look at the guide: ECS Operator

Parameters

  • task_definition (str) -- the task definition name on Elastic Container Service

  • cluster (str) -- the cluster name on Elastic Container Service

  • overrides (dict) -- the same parameter that boto3 will receive (templated): https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html#ECS.Client.run_task

  • aws_conn_id (str) -- connection id of AWS credentials / region name. If None, credential boto3 strategy will be used (http://boto3.readthedocs.io/en/latest/guide/configuration.html).

  • region_name (str) -- region name to use in AWS Hook. Override the region_name in connection (if provided)

  • launch_type (str) -- the launch type on which to run your task ('EC2' or 'FARGATE')

  • group (str) -- the name of the task group associated with the task

  • placement_constraints (list) -- an array of placement constraint objects to use for the task

  • placement_strategy (list) -- an array of placement strategy objects to use for the task

  • platform_version (str) -- the platform version on which your task is running

  • network_configuration (dict) -- the network configuration for the task

  • tags (dict) -- a dictionary of tags in the form of {'tagKey': 'tagValue'}.

  • awslogs_group (str) -- the CloudWatch group where your ECS container logs are stored. Only required if you want logs to be shown in the Airflow UI after your job has finished.

  • awslogs_region (str) -- the region in which your CloudWatch logs are stored. If None, this is the same as the region_name parameter. If that is also None, this is the default AWS region based on your connection settings.

  • awslogs_stream_prefix (str) -- the stream prefix that is used for the CloudWatch logs. This is usually based on some custom name combined with the name of the container. Only required if you want logs to be shown in the Airflow UI after your job has finished.

  • reattach (bool) -- If set to True, will check if a task from the same family is already running. If so, the operator will attach to it instead of starting a new task.

ui_color = #f0ede4[source]
template_fields = ['overrides'][source]
execute(self, context)[source]
_start_task(self)[source]
_try_reattach_task(self)[source]
_wait_for_task_ended(self)[source]
_cloudwatch_log_events(self)[source]
_aws_logs_enabled(self)[source]
_last_log_message(self)[source]
_check_success_task(self)[source]
get_hook(self)[source]

Create and return an AwsHook.

get_logs_hook(self)[source]

Create and return an AwsLogsHook.

on_kill(self)[source]

Was this entry helpful?