airflow.providers.amazon.aws.operators.glue_databrew

Module Contents

Classes

GlueDataBrewStartJobOperator

Start an AWS Glue DataBrew job.
class airflow.providers.amazon.aws.operators.glue_databrew.GlueDataBrewStartJobOperator(job_name, wait_for_completion=True, delay=30, deferrable=conf.getboolean('operators', 'default_deferrable', fallback=False), aws_conn_id='aws_default', **kwargs)[source]

Bases: airflow.models.BaseOperator

Start an AWS Glue DataBrew job.

AWS Glue DataBrew is a visual data preparation tool that makes it easier for data analysts and data scientists to clean and normalize data to prepare it for analytics and machine learning (ML).

See also

For more information on how to use this operator, take a look at the guide: Start an AWS Glue DataBrew job

Parameters

  • job_name (str) – unique job name per AWS Account

  • wait_for_completion (bool) – Whether to wait for job run completion. (default: True)

  • deferrable (bool) – If True, the operator will wait asynchronously for the job to complete. This implies waiting for completion. This mode requires aiobotocore module to be installed. (default: False)

  • delay (int) – Time in seconds to wait between status checks. Default is 30.

Returns

dictionary with key run_id and value of the resulting job’s run_id.

template_fields: Sequence[str] = ('job_name', 'wait_for_completion', 'delay', 'deferrable')[source]
hook()[source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

execute_complete(context, event=None)[source]

Was this entry helpful?