airflow.contrib.operators.awsbatch_operator

Module Contents

class airflow.contrib.operators.awsbatch_operator.AWSBatchOperator(job_name, job_definition, job_queue, overrides, array_properties=None, parameters=None, max_retries=MAX_RETRIES, status_retries=STATUS_RETRIES, aws_conn_id=None, region_name=None, **kwargs)[source]

Bases: airflow.models.BaseOperator

Execute a job on AWS Batch Service

Parameters
MAX_RETRIES = 4200[source]
STATUS_RETRIES = 10[source]
ui_color = #c3dae0[source]
client[source]
arn[source]
template_fields = ['job_name', 'overrides', 'parameters'][source]
execute(self, context)[source]
_wait_for_task_ended(self)[source]

Try to use a waiter from the below pull request

If the waiter is not available apply a exponential backoff

  • docs.aws.amazon.com/general/latest/gr/api-retries.html

_poll_for_task_ended(self)[source]

Poll for job status

  • docs.aws.amazon.com/general/latest/gr/api-retries.html

_get_job_description(self)[source]

Get job description

_check_success_task(self)[source]

Check the final status of the batch job; the job status options are: ‘SUBMITTED’|’PENDING’|’RUNNABLE’|’STARTING’|’RUNNING’|’SUCCEEDED’|’FAILED’

get_hook(self)[source]
on_kill(self)[source]

Was this entry helpful?