Module Contents



Waits for a partition to show up in AWS Glue Catalog.


This sensor is deprecated. Please use

class*, table_name, expression="ds='{{ ds }}'", aws_conn_id='aws_default', region_name=None, database_name='default', poke_interval=60 * 3, **kwargs)[source]

Bases: airflow.sensors.base.BaseSensorOperator

Waits for a partition to show up in AWS Glue Catalog.

  • table_name (str) -- The name of the table to wait for, supports the dot notation (my_database.my_table)

  • expression (str) -- The partition clause to wait for. This is passed as is to the AWS Glue Catalog API's get_partitions function, and supports SQL like notation as in ds='2015-01-01' AND type='value' and comparison operators as in "ds>=2015-01-01". See #aws-glue-api-catalog-partitions-GetPartitions

  • aws_conn_id (str) -- ID of the Airflow connection where credentials and extra configuration are stored

  • region_name (Optional[str]) -- Optional aws region name (example: us-east-1). Uses region from connection if not specified.

  • database_name (str) -- The name of the catalog database where the partitions reside.

  • poke_interval (int) -- Time in seconds that the job should wait in between each tries

template_fields :Sequence[str] = ['database_name', 'table_name', 'expression'][source]
ui_color = #C5CAE9[source]
poke(self, context)[source]

Checks for existence of the partition in the AWS Glue Catalog table


Gets the GlueCatalogHook

class*args, **kwargs)[source]

Bases: GlueCatalogPartitionSensor

This sensor is deprecated. Please use

Was this entry helpful?