airflow.providers.apache.hive.macros.hive
¶
Module Contents¶
Functions¶
|
Get the max partition for a table. |
|
Find the date in a list closest to the target date. |
- airflow.providers.apache.hive.macros.hive.max_partition(table, schema='default', field=None, filter_map=None, metastore_conn_id='metastore_default')[source]¶
Get the max partition for a table.
- Parameters
schema – The hive schema the table lives in
table – The hive table you are interested in, supports the dot notation as in “my_database.my_table”, if a dot is found, the schema param is disregarded
metastore_conn_id – The hive connection you are interested in. If your default is set you don’t need to use this parameter.
filter_map – partition_key:partition_value map used for partition filtering, e.g. {‘key1’: ‘value1’, ‘key2’: ‘value2’}. Only partitions matching all partition_key:partition_value pairs will be considered as candidates of max partition.
field – the field to get the max value from. If there’s only one partition field, this will be inferred
>>> max_partition("airflow.static_babynames_partitioned") '2015-01-01'
- airflow.providers.apache.hive.macros.hive.closest_ds_partition(table, ds, before=True, schema='default', metastore_conn_id='metastore_default')[source]¶
Find the date in a list closest to the target date.
An optional parameter can be given to get the closest before or after.
- Parameters
table – A hive table name
ds – A datestamp
%Y-%m-%d
e.g.yyyy-mm-dd
before – closest before (True), after (False) or either side of ds
schema – table schema
metastore_conn_id – which metastore connection to use
- Returns
The closest date
- Return type
str | None
>>> tbl = "airflow.static_babynames_partitioned" >>> closest_ds_partition(tbl, "2015-01-02") '2015-01-01'