airflow.providers.amazon.aws.operators.glue_catalog

AWS Glue Data Catalog operators.

Classes

GlueCatalogCreateDatabaseOperator

Create a database in the AWS Glue Data Catalog.

GlueCatalogDeleteDatabaseOperator

Delete a database from the AWS Glue Data Catalog.

GlueCatalogCreateTableOperator

Create a table in an AWS Glue Data Catalog database.

GlueCatalogDeleteTableOperator

Delete a table from an AWS Glue Data Catalog database.

Module Contents

class airflow.providers.amazon.aws.operators.glue_catalog.GlueCatalogCreateDatabaseOperator(*, database_name, description=None, location_uri=None, parameters=None, catalog_id=None, tags=None, if_exists='skip', **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook]

Create a database in the AWS Glue Data Catalog.

See also

For more information on how to use this operator, take a look at the guide: Create a Catalog Database

Parameters:

  • database_name (str) – The name of the database to create.

  • description (str | None) – A description of the database.

  • location_uri (str | None) – The location of the database (e.g. an S3 path).

  • parameters (dict[str, str] | None) – Key-value pairs that define properties of the database.

  • catalog_id (str | None) – The ID of the Data Catalog. Defaults to the account ID.

  • tags (dict[str, str] | None) – Tags to assign to the database.

  • if_exists (Literal['fail', 'skip']) – Behavior when the database already exists. "fail" raises an error, "skip" logs and returns the database name.

aws_hook_class[source]
template_fields: tuple[str, Ellipsis][source]
database_name[source]
description = None[source]
location_uri = None[source]
parameters = None[source]
catalog_id = None[source]
tags = None[source]
if_exists = 'skip'[source]
execute(context)[source]

Derive when creating an operator.

The main method to execute the task. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.amazon.aws.operators.glue_catalog.GlueCatalogDeleteDatabaseOperator(*, database_name, catalog_id=None, **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook]

Delete a database from the AWS Glue Data Catalog.

See also

For more information on how to use this operator, take a look at the guide: Delete a Catalog Database

Parameters:

  • database_name (str) – The name of the database to delete.

  • catalog_id (str | None) – The ID of the Data Catalog. Defaults to the account ID.

aws_hook_class[source]
template_fields: tuple[str, Ellipsis][source]
database_name[source]
catalog_id = None[source]
execute(context)[source]

Derive when creating an operator.

The main method to execute the task. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.amazon.aws.operators.glue_catalog.GlueCatalogCreateTableOperator(*, database_name, table_name, table_input, catalog_id=None, if_exists='skip', **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook]

Create a table in an AWS Glue Data Catalog database.

See also

For more information on how to use this operator, take a look at the guide: Create a Table

Parameters:

  • database_name (str) – The name of the database. (templated)

  • table_name (str) – The name of the table. (templated)

  • table_input (dict[str, Any]) – The TableInput dict defining the table schema, storage, etc. (templated)

  • catalog_id (str | None) – The ID of the Data Catalog. Defaults to the account ID. (templated)

  • if_exists (Literal['fail', 'skip']) – Behavior when the table already exists. "fail" raises an error, "skip" logs and returns.

aws_hook_class[source]
template_fields: tuple[str, Ellipsis][source]
template_fields_renderers[source]
database_name[source]
table_name[source]
table_input[source]
catalog_id = None[source]
if_exists = 'skip'[source]
execute(context)[source]

Derive when creating an operator.

The main method to execute the task. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

class airflow.providers.amazon.aws.operators.glue_catalog.GlueCatalogDeleteTableOperator(*, database_name, table_name, catalog_id=None, **kwargs)[source]

Bases: airflow.providers.amazon.aws.operators.base_aws.AwsBaseOperator[airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook]

Delete a table from an AWS Glue Data Catalog database.

See also

For more information on how to use this operator, take a look at the guide: Delete a Table

Parameters:

  • database_name (str) – The name of the database. (templated)

  • table_name (str) – The name of the table to delete. (templated)

  • catalog_id (str | None) – The ID of the Data Catalog. Defaults to the account ID. (templated)

aws_hook_class[source]
template_fields: tuple[str, Ellipsis][source]
database_name[source]
table_name[source]
catalog_id = None[source]
execute(context)[source]

Derive when creating an operator.

The main method to execute the task. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

Was this entry helpful?