tests.system.providers.google.cloud.dataproc_metastore.example_dataproc_metastore_hive_partition_sensor¶
Example Airflow DAG that show how to check Hive partitions existence using Dataproc Metastore Sensor.
Note that Metastore service must be configured to use gRPC endpoints,
Module Contents¶
- tests.system.providers.google.cloud.dataproc_metastore.example_dataproc_metastore_hive_partition_sensor.DAG_ID = 'dataproc_metastore_hive_partition_sensor'[source]¶
- tests.system.providers.google.cloud.dataproc_metastore.example_dataproc_metastore_hive_partition_sensor.PROJECT_ID[source]¶
- tests.system.providers.google.cloud.dataproc_metastore.example_dataproc_metastore_hive_partition_sensor.ENV_ID[source]¶
- tests.system.providers.google.cloud.dataproc_metastore.example_dataproc_metastore_hive_partition_sensor.SERVICE_ID[source]¶
- tests.system.providers.google.cloud.dataproc_metastore.example_dataproc_metastore_hive_partition_sensor.REGION = 'europe-west1'[source]¶
- tests.system.providers.google.cloud.dataproc_metastore.example_dataproc_metastore_hive_partition_sensor.TABLE_NAME = 'test_table'[source]¶
- tests.system.providers.google.cloud.dataproc_metastore.example_dataproc_metastore_hive_partition_sensor.PARTITION_1 = 'column1=value1'[source]¶
- tests.system.providers.google.cloud.dataproc_metastore.example_dataproc_metastore_hive_partition_sensor.PARTITION_2 = 'column2=value2/column3=value3'[source]¶