tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group¶

Example Airflow DAG for Google Cloud Managed Service for Apache Kafka testing Topic operations.

Requirements:

Operator to create a cluster requires GOOGLE_PROVIDER_NETWORK environmental variable that will contain the name of the network that will be used for cluster creation.

Please, note that if you are running this operator in Google Cloud Composer, this value will be set automatically and will not require any additional configuration. In other cases, the network in which the cluster will be created should be the same as your machine is running in.

Attributes¶

`ENV_ID`
`PROJECT_ID`
`IS_COMPOSER`
`NETWORK`
`DAG_ID`
`LOCATION`
`CLUSTER_ID`
`CLUSTER_CONF`
`TOPIC_ID`
`TOPIC_CONF`
`CONSUMER_GROUP_ID`
`CONNECTION_ID`
`PORT`
`BOOTSTRAP_URL`
`log`
`create_cluster`
`test_run`

Functions¶

`producer`()	Produce and submit 10 messages
`consumer`(message)	Take in consumed messages and print its contents to the logs.

Module Contents¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.ENV_ID[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.PROJECT_ID[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.IS_COMPOSER[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.NETWORK[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.DAG_ID = 'managed_kafka_consumer_group_operations'[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.LOCATION = 'us-central1'[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.CLUSTER_ID = ''[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.CLUSTER_CONF[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.TOPIC_ID = ''[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.TOPIC_CONF[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.CONSUMER_GROUP_ID = ''[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.CONNECTION_ID = 'connection_managed_kafka_consumer_group_operations_Uninferable'[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.PORT = '9092'[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.BOOTSTRAP_URL[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.log[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.producer()[source]¶: Produce and submit 10 messages

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.consumer(message)[source]¶: Take in consumed messages and print its contents to the logs.

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.create_cluster[source]¶

tests.system.google.cloud.managed_kafka.example_managed_kafka_consumer_group.test_run[source]¶