airflow.providers.openlineage.plugins.facets

Module Contents

Classes

AirflowMappedTaskRunFacet

Run facet containing information about mapped tasks.

AirflowJobFacet

Composite Airflow job facet.

AirflowStateRunFacet

Airflow facet providing state information.

AirflowRunFacet

Composite Airflow run facet.

AirflowDagRunFacet

Composite Airflow DAG run facet.

AirflowDebugRunFacet

Airflow Debug run facet.

UnknownOperatorInstance

Describes an unknown operator.

UnknownOperatorAttributeRunFacet

RunFacet that describes unknown operators in an Airflow DAG.

class airflow.providers.openlineage.plugins.facets.AirflowMappedTaskRunFacet[source]

Bases: openlineage.client.facet_v2.RunFacet

Run facet containing information about mapped tasks.

mapIndex: int[source]
operatorClass: str[source]
classmethod from_task_instance(task_instance)[source]
class airflow.providers.openlineage.plugins.facets.AirflowJobFacet[source]

Bases: openlineage.client.facet_v2.JobFacet

Composite Airflow job facet.

This facet encapsulates all the necessary information to re-create full scope of an Airflow DAG logic, enabling reconstruction, visualization, and analysis of DAGs in a comprehensive manner. It includes detailed representations of the tasks, task groups, and their hierarchical relationships, making it possible to draw a graph that visually represents the entire DAG structure (like in Airflow UI). It also indicates whether a task should emit an OpenLineage (OL) event, enabling consumers to anticipate the number of events and identify the tasks from which they can expect these events.

Attributes:

taskTree: A dictionary representing the hierarchical structure of tasks in the DAG. taskGroups: A dictionary that contains information about task groups within the DAG. tasks: A dictionary detailing individual tasks within the DAG.

taskTree: dict[source]
taskGroups: dict[source]
tasks: dict[source]
class airflow.providers.openlineage.plugins.facets.AirflowStateRunFacet[source]

Bases: openlineage.client.facet_v2.RunFacet

Airflow facet providing state information.

This facet is designed to be sent at a completion event, offering state information about the DAG run and each individual task. This information is crucial for understanding the execution flow and comprehensive post-run analysis and debugging, including why certain tasks did not emit events, which can occur due to the use of control flow operators like the BranchOperator.

Attributes:

dagRunState: This indicates the final status of the entire DAG run (e.g., “success”, “failed”). tasksState: A dictionary mapping task IDs to their respective states. (e.g., “failed”, “skipped”).

dagRunState: str[source]
tasksState: dict[str, str][source]
class airflow.providers.openlineage.plugins.facets.AirflowRunFacet[source]

Bases: openlineage.client.facet_v2.RunFacet

Composite Airflow run facet.

dag: dict[source]
dagRun: dict[source]
task: dict[source]
taskInstance: dict[source]
taskUuid: str[source]
class airflow.providers.openlineage.plugins.facets.AirflowDagRunFacet[source]

Bases: openlineage.client.facet_v2.RunFacet

Composite Airflow DAG run facet.

dag: dict[source]
dagRun: dict[source]
class airflow.providers.openlineage.plugins.facets.AirflowDebugRunFacet[source]

Bases: openlineage.client.facet_v2.RunFacet

Airflow Debug run facet.

packages: dict[source]
class airflow.providers.openlineage.plugins.facets.UnknownOperatorInstance[source]

Bases: openlineage.client.utils.RedactMixin

Describes an unknown operator.

This specifies the (class) name of the operator and its properties.

name: str[source]
properties: dict[str, object][source]
type: str = 'operator'[source]
class airflow.providers.openlineage.plugins.facets.UnknownOperatorAttributeRunFacet[source]

Bases: openlineage.client.facet_v2.RunFacet

RunFacet that describes unknown operators in an Airflow DAG.

unknownItems: list[UnknownOperatorInstance][source]

Was this entry helpful?