airflow.providers.openlineage.extractors.base

Module Contents

Classes

OperatorLineage

Structure returned from lineage extraction.

BaseExtractor

Abstract base extractor class.

DefaultExtractor

Extractor that uses get_openlineage_facets_on_start/complete/failure methods.

Attributes

DatasetSubclass

BaseFacetSubclass

airflow.providers.openlineage.extractors.base.DatasetSubclass[source]
airflow.providers.openlineage.extractors.base.BaseFacetSubclass[source]
class airflow.providers.openlineage.extractors.base.OperatorLineage[source]

Bases: Generic[DatasetSubclass, BaseFacetSubclass]

Structure returned from lineage extraction.

inputs: list[DatasetSubclass][source]
outputs: list[DatasetSubclass][source]
run_facets: dict[str, BaseFacetSubclass][source]
job_facets: dict[str, BaseFacetSubclass][source]
class airflow.providers.openlineage.extractors.base.BaseExtractor(operator)[source]

Bases: abc.ABC, airflow.utils.log.logging_mixin.LoggingMixin

Abstract base extractor class.

This is used mostly to maintain support for custom extractors.

abstract classmethod get_operator_classnames()[source]

Get a list of operators that extractor works for.

This is an abstract method that subclasses should implement. There are operators that work very similarly and one extractor can cover.

extract()[source]
extract_on_complete(task_instance)[source]
class airflow.providers.openlineage.extractors.base.DefaultExtractor(operator)[source]

Bases: BaseExtractor

Extractor that uses get_openlineage_facets_on_start/complete/failure methods.

classmethod get_operator_classnames()[source]

Assign this extractor to no operators.

Default extractor is chosen not on the classname basis, but by existence of get_openlineage_facets method on operator.

extract_on_complete(task_instance)[source]

Was this entry helpful?