airflow.providers.openlineage.utils.spark¶
Attributes¶
Functions¶
Inject parent job information into Spark properties if not already present. |
|
Inject transport information into Spark properties if not already present. |
Module Contents¶
- airflow.providers.openlineage.utils.spark.inject_parent_job_information_into_spark_properties(properties, context)[source]¶
Inject parent job information into Spark properties if not already present.
- Args:
properties: Spark properties. context: The context containing task instance information.
- Returns:
Modified Spark properties with OpenLineage parent job information properties injected, if applicable.
- airflow.providers.openlineage.utils.spark.inject_transport_information_into_spark_properties(properties, context)[source]¶
Inject transport information into Spark properties if not already present.
- Args:
properties: Spark properties. context: The context containing task instance information.
- Returns:
Modified Spark properties with OpenLineage transport information properties injected, if applicable.