Configuration Reference¶

This page contains the list of all available Airflow configurations for the apache-airflow-providers-openlineage provider that can be set in the airflow.cfg file or using environment variables.

Note

The configuration embedded in provider packages started to be used as of Airflow 2.7.0. Previously the configuration was described and configured in the Airflow core package - so if you are using Airflow below 2.7.0, look at Airflow documentation for the list of available configuration options that were available in Airflow core.

Note

For more information see Setting Configuration Options.

Sections:

[openlineage]

[openlineage]¶

This section applies settings for OpenLineage integration. More about configuration and it’s precedence can be found at https://airflow.apache.org/docs/apache-airflow-providers-openlineage/stable/guides/user.html#transport-setup

config_path¶

Specify the path to the YAML configuration file. This ensures backwards compatibility with passing config through the openlineage.yml file.

Type: string
Default: ''
Environment Variable: AIRFLOW__OPENLINEAGE__CONFIG_PATH
Example: full/path/to/openlineage.yml

disable_source_code¶

Disable the inclusion of source code in OpenLineage events by setting this to true. By default, several Operators (e.g. Python, Bash) will include their source code in the events unless disabled.

Type: boolean
Default: None
Environment Variable: AIRFLOW__OPENLINEAGE__DISABLE_SOURCE_CODE

disabled¶

Disable sending events without uninstalling the OpenLineage Provider by setting this to true.

Type: boolean
Default: False
Environment Variable: AIRFLOW__OPENLINEAGE__DISABLED

disabled_for_operators¶

New in version 1.1.0.

Exclude some Operators from emitting OpenLineage events by passing a string of semicolon separated full import paths of Operators to disable.

Type: string
Default: ''
Environment Variable: AIRFLOW__OPENLINEAGE__DISABLED_FOR_OPERATORS
Example: airflow.operators.bash.BashOperator;airflow.operators.python.PythonOperator

extractors¶

Register custom OpenLineage Extractors by passing a string of semicolon separated full import paths.

Type: string
Default: None
Environment Variable: AIRFLOW__OPENLINEAGE__EXTRACTORS
Example: full.path.to.ExtractorClass;full.path.to.AnotherExtractorClass

namespace¶

Set namespace that the lineage data belongs to, so that if you use multiple OpenLineage producers, events coming from them will be logically separated.

Type: string
Default: None
Environment Variable: AIRFLOW__OPENLINEAGE__NAMESPACE
Example: my_airflow_instance_1

selective_enable¶

New in version 1.7.0.

If this setting is enabled, OpenLineage integration won’t collect and emit metadata, unless you explicitly enable it per DAG or Task using enable_lineage method.

Type: boolean
Default: False
Environment Variable: AIRFLOW__OPENLINEAGE__SELECTIVE_ENABLE

transport¶

Pass OpenLineage Client transport configuration as JSON string. It should contain type of the transport and additional options (different for each transport type). For more details see: https://openlineage.io/docs/client/python/#built-in-transport-types

Currently supported types are:

HTTP

Kafka

Console

File

Type: string
Default: ''
Environment Variable: AIRFLOW__OPENLINEAGE__TRANSPORT
Example: {"type": "http", "url": "http://localhost:5000", "endpoint": "api/v1/lineage"}