Airflow Summit 2025 is coming October 07-09. Register now to secure your spot!

tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore

Example Airflow DAG that show how to use various Dataproc Metastore operators to manage a service.

Attributes

DAG_ID

PROJECT_ID

ENV_ID

SERVICE_ID

METADATA_IMPORT_ID

REGION

RESOURCE_DATA_BUCKET

BUCKET_NAME

TIMEOUT

DB_TYPE

DESTINATION_GCS_FOLDER

HIVE_FILE

GCS_URI

SERVICE

METADATA_IMPORT

SERVICE_TO_UPDATE

UPDATE_MASK

create_bucket

test_run

Module Contents

tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.DAG_ID = 'dataproc_metastore'[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.PROJECT_ID[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.ENV_ID[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.SERVICE_ID = ''[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.METADATA_IMPORT_ID = ''[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.REGION = 'us-central1'[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.RESOURCE_DATA_BUCKET = 'airflow-system-tests-resources'[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.BUCKET_NAME = 'bucket_dataproc_metastore_Uninferable'[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.TIMEOUT = 2400[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.DB_TYPE = 'MYSQL'[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.DESTINATION_GCS_FOLDER = 'gs://bucket_dataproc_metastore_Uninferable/>'[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.HIVE_FILE = 'hive.sql'[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.GCS_URI = 'gs://bucket_dataproc_metastore_Uninferable/dataproc/hive.sql'[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.SERVICE[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.METADATA_IMPORT[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.SERVICE_TO_UPDATE[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.UPDATE_MASK[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.create_bucket[source]
tests.system.google.cloud.dataproc_metastore.example_dataproc_metastore.test_run[source]

Was this entry helpful?