Configuration Reference¶
This page contains the list of all available Airflow configurations for the
apache-airflow-providers-common-io provider that can be set in the airflow.cfg file or using environment variables.
Note
The configuration embedded in provider packages started to be used as of Airflow 2.7.0. Previously the configuration was described and configured in the Airflow core package - so if you are using Airflow below 2.7.0, look at Airflow documentation for the list of available configuration options that were available in Airflow core.
Note
For more information see Setting Configuration Options.
Sections:
[common.io]¶
Common IO configuration section
xcom_objectstorage_compression¶
New in version 1.3.0.
Compression algorithm to use when storing XComs in object storage. Supported algorithms are a.o.: snappy, zip, gzip, bz2, and lzma. If not specified, no compression will be used. Note that the compression algorithm must be available in the Python installation (e.g. python-snappy for snappy). Zip, gz, bz2 are available by default.
- Type
string
- Default
''- Environment Variable
AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_COMPRESSION- Example
gz
xcom_objectstorage_path¶
New in version 1.3.0.
Path to a location on object storage where XComs can be stored in url format.
- Type
string
- Default
''- Environment Variable
AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_PATH- Example
s3://conn_id@bucket/path
xcom_objectstorage_threshold¶
New in version 1.3.0.
Threshold in bytes for storing XComs in object storage. -1 means always store in the database. 0 means always store in object storage. Any positive number means it will be stored in object storage if the size of the value is greater than the threshold.
- Type
integer
- Default
-1- Environment Variable
AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_THRESHOLD- Example
1000000