airflow.providers.amazon.aws.operators.glue_crawler

Module Contents

Classes

GlueCrawlerOperator

Creates, updates and triggers an AWS Glue Crawler. AWS Glue Crawler is a serverless

AwsGlueCrawlerOperator

This operator is deprecated.
class airflow.providers.amazon.aws.operators.glue_crawler.GlueCrawlerOperator(config, aws_conn_id='aws_default', poll_interval: int = 5, **kwargs)[source]

Bases: airflow.models.BaseOperator

Creates, updates and triggers an AWS Glue Crawler. AWS Glue Crawler is a serverless service that manages a catalog of metadata tables that contain the inferred schema, format and data types of data stores within the AWS cloud.

Parameters

  • config (dict) -- Configurations for the AWS Glue crawler

  • aws_conn_id (Optional[str]) -- aws connection to use

  • poll_interval (Optional[int]) -- Time (in seconds) to wait between two consecutive calls to check crawler status

ui_color = #ededed[source]
hook(self) → airflow.providers.amazon.aws.hooks.glue_crawler.GlueCrawlerHook[source]

Create and return an GlueCrawlerHook.

execute(self, context: airflow.utils.context.Context)[source]

Executes AWS Glue Crawler from Airflow

Returns

the name of the current glue crawler.

class airflow.providers.amazon.aws.operators.glue_crawler.AwsGlueCrawlerOperator(*args, **kwargs)[source]

Bases: GlueCrawlerOperator

This operator is deprecated. Please use airflow.providers.amazon.aws.operators.glue_crawler.GlueCrawlerOperator.

Was this entry helpful?