Airflow Summit 2025 is coming October 07-09. Register now for early bird ticket!

tests.system.google.cloud.dataflow.example_dataflow_streaming_python¶

Example Airflow DAG for testing Google Dataflow Beam Pipeline Operator with Python for Streaming job.

Attributes¶

`ENV_ID`
`PROJECT_ID`
`DAG_ID`
`RESOURCE_DATA_BUCKET`
`BUCKET_NAME`
`GCS_TMP`
`GCS_STAGING`
`GCS_PYTHON_SCRIPT`
`LOCATION`
`TOPIC_ID`
`TOPIC_ID_2`
`default_args`
`create_bucket`
`test_run`

Module Contents¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.ENV_ID[source]¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.PROJECT_ID[source]¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.DAG_ID = 'dataflow_native_python_streaming'[source]¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.RESOURCE_DATA_BUCKET = 'airflow-system-tests-resources'[source]¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.BUCKET_NAME = 'bucket_dataflow_native_python_streaming_Uninferable'[source]¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.GCS_TMP = 'gs://bucket_dataflow_native_python_streaming_Uninferable/temp/'[source]¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.GCS_STAGING = 'gs://bucket_dataflow_native_python_streaming_Uninferable/staging/'[source]¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.GCS_PYTHON_SCRIPT = 'gs://airflow-system-tests-resources/dataflow/python/streaming_wordcount.py'[source]¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.LOCATION = 'europe-west3'[source]¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.TOPIC_ID = 'topic-dataflow_native_python_streaming'[source]¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.TOPIC_ID_2 = 'topic-2-dataflow_native_python_streaming'[source]¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.default_args[source]¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.create_bucket[source]¶

tests.system.google.cloud.dataflow.example_dataflow_streaming_python.test_run[source]¶

Was this entry helpful?