airflow.providers.apache.kafka.operators.produce
¶
Module Contents¶
Classes¶
An operator that produces messages to a Kafka topic |
Attributes¶
- class airflow.providers.apache.kafka.operators.produce.ProduceToTopicOperator(topic, producer_function, kafka_config_id='kafka_default', producer_function_args=None, producer_function_kwargs=None, delivery_callback=None, synchronous=True, poll_timeout=0, **kwargs)[source]¶
Bases:
airflow.models.BaseOperator
An operator that produces messages to a Kafka topic
Registers a producer to a kafka topic and publishes messages to the log.
- Parameters
kafka_config_id (str) – The connection object to use, defaults to “kafka_default”
topic (str) – The topic the producer should produce to, defaults to None
producer_function (str | Callable[..., Any]) – The function that generates key/value pairs as messages for production, defaults to None
producer_function_args (Sequence[Any] | None) – Additional arguments to be applied to the producer callable, defaults to None
producer_function_kwargs (dict[Any, Any] | None) – Additional keyword arguments to be applied to the producer callable, defaults to None
delivery_callback (str | None) – The callback to apply after delivery(or failure) of a message, defaults to None
synchronous (bool) – If writes to kafka should be fully synchronous, defaults to True
poll_timeout (float) – How long of a delay should be applied when calling poll after production to kafka, defaults to 0
- Raises
AirflowException – _description_
See also
For more information on how to use this operator, take a look at the guide: ProduceToTopicOperator