airflow.providers.apache.hive.sensors.hive_partition
¶
Module Contents¶
Classes¶
Waits for a partition to show up in Hive. |
- class airflow.providers.apache.hive.sensors.hive_partition.HivePartitionSensor(*, table, partition="ds='{{ ds }}'", metastore_conn_id='metastore_default', schema='default', poke_interval=60 * 3, **kwargs)[source]¶
Bases:
airflow.sensors.base.BaseSensorOperator
Waits for a partition to show up in Hive.
Note: Because
partition
supports general logical operators, it can be inefficient. Consider using NamedHivePartitionSensor instead if you don’t need the full flexibility of HivePartitionSensor.- Parameters
table (str) – The name of the table to wait for, supports the dot notation (my_database.my_table)
partition (str | None) – The partition clause to wait for. This is passed as is to the metastore Thrift client
get_partitions_by_filter
method, and apparently supports SQL like notation as inds='2015-01-01' AND type='value'
and comparison operators as in"ds>=2015-01-01"
metastore_conn_id (str) – reference to the :ref: metastore thrift service connection id <howto/connection:hive_metastore>