tests.system.google.cloud.dataproc.example_dataproc_batch_persistent¶

Example Airflow DAG for Managed Spark batch operators.

Attributes¶

`ENV_ID`
`DAG_ID`
`PROJECT_ID`
`BUCKET_NAME`
`REGION`
`CLUSTER_NAME_BASE`
`CLUSTER_NAME_FULL`
`CLUSTER_NAME`
`BATCH_ID`
`CLUSTER_GENERATOR_CONFIG_FOR_PHS`
`BATCH_CONFIG_WITH_PHS`
`create_bucket`
`test_run`

tests.system.google.cloud.dataproc.example_dataproc_batch_persistent.ENV_ID[source]¶

tests.system.google.cloud.dataproc.example_dataproc_batch_persistent.DAG_ID = 'dataproc_batch_ps'[source]¶

tests.system.google.cloud.dataproc.example_dataproc_batch_persistent.PROJECT_ID[source]¶

tests.system.google.cloud.dataproc.example_dataproc_batch_persistent.BUCKET_NAME[source]¶

tests.system.google.cloud.dataproc.example_dataproc_batch_persistent.REGION = 'us-east4'[source]¶

tests.system.google.cloud.dataproc.example_dataproc_batch_persistent.CLUSTER_NAME_BASE = ''[source]¶

tests.system.google.cloud.dataproc.example_dataproc_batch_persistent.CLUSTER_NAME_FULL[source]¶

tests.system.google.cloud.dataproc.example_dataproc_batch_persistent.CLUSTER_NAME = ''[source]¶

tests.system.google.cloud.dataproc.example_dataproc_batch_persistent.BATCH_ID[source]¶

tests.system.google.cloud.dataproc.example_dataproc_batch_persistent.CLUSTER_GENERATOR_CONFIG_FOR_PHS[source]¶

tests.system.google.cloud.dataproc.example_dataproc_batch_persistent.BATCH_CONFIG_WITH_PHS[source]¶

tests.system.google.cloud.dataproc.example_dataproc_batch_persistent.create_bucket[source]¶

tests.system.google.cloud.dataproc.example_dataproc_batch_persistent.test_run[source]¶