airflow.providers.openlineage.extractors.base

Module Contents

Classes

OperatorLineage

Structure returned from lineage extraction.

BaseExtractor

Abstract base extractor class.

DefaultExtractor

Extractor that uses get_openlineage_facets_on_start/complete/failure methods.
class airflow.providers.openlineage.extractors.base.OperatorLineage[source]

Structure returned from lineage extraction.

inputs: list[openlineage.client.run.Dataset][source]
outputs: list[openlineage.client.run.Dataset][source]
run_facets: dict[str, openlineage.client.facet.BaseFacet][source]
job_facets: dict[str, openlineage.client.facet.BaseFacet][source]
class airflow.providers.openlineage.extractors.base.BaseExtractor(operator)[source]

Bases: abc.ABC, airflow.utils.log.logging_mixin.LoggingMixin

Abstract base extractor class.

This is used mostly to maintain support for custom extractors.

abstract classmethod get_operator_classnames()[source]

Get a list of operators that extractor works for.

This is an abstract method that subclasses should implement. There are operators that work very similarly and one extractor can cover.

validate()[source]
abstract extract()[source]
extract_on_complete(task_instance)[source]
class airflow.providers.openlineage.extractors.base.DefaultExtractor(operator)[source]

Bases: BaseExtractor

Extractor that uses get_openlineage_facets_on_start/complete/failure methods.

classmethod get_operator_classnames()[source]

Assign this extractor to no operators.

Default extractor is chosen not on the classname basis, but by existence of get_openlineage_facets method on operator.

extract()[source]
extract_on_complete(task_instance)[source]

Was this entry helpful?