An alternative to the HivePartitionSensor that talk directly to the
- class airflow.providers.apache.hive.sensors.metastore_partition.MetastorePartitionSensor(*, table, partition_name, schema='default', mysql_conn_id='metastore_mysql', **kwargs)¶
An alternative to the HivePartitionSensor that talk directly to the MySQL db. This was created as a result of observing sub optimal queries generated by the Metastore thrift service when hitting subpartitioned tables. The Thrift service’s queries were written in a way that would not leverage the indexes.
schema (str) – the schema
table (str) – the table
partition_name (str) – the partition name, as defined in the PARTITIONS table of the Metastore. Order of the fields does matter. Examples:
ds=2016-01-01/sub=foofor a sub partitioned table
mysql_conn_id (str) – a reference to the MySQL conn_id for the metastore
- template_fields :Sequence[str] = ['partition_name', 'table', 'schema']¶
- ui_color = #8da7be¶
Function defined by the sensors while deriving this class should override.