Configuration Reference¶

This page contains the list of all available Airflow configurations for the apache-airflow-providers-opensearch provider that can be set in the airflow.cfg file or using environment variables.

Note

The configuration embedded in provider packages started to be used as of Airflow 2.7.0. Previously the configuration was described and configured in the Airflow core package - so if you are using Airflow below 2.7.0, look at Airflow documentation for the list of available configuration options that were available in Airflow core.

Note

For more information see Setting Configuration Options.

Sections:

[opensearch]
[opensearch_configs]

[opensearch]¶

end_of_log_mark¶

Used to mark the end of a log stream for a task

Type: string
Default: end_of_log
Environment Variable: AIRFLOW__OPENSEARCH__END_OF_LOG_MARK

host¶

Opensearch host

Type: string
Default: ''
Environment Variable: AIRFLOW__OPENSEARCH__HOST

host_field¶

The field where host name is stored (normally either host or host.name)

Type: string
Default: host
Environment Variable: AIRFLOW__OPENSEARCH__HOST_FIELD

index_patterns¶

Comma separated list of index patterns to use when searching for logs (default: _all). The index_patterns_callable takes precedence over this.

Type: string
Default: _all
Environment Variable: AIRFLOW__OPENSEARCH__INDEX_PATTERNS
Example: something-*

index_patterns_callable¶

A string representing the full path to the Python callable path which accept TI object and return comma separated list of index patterns. This will takes precedence over index_patterns.

Type: string
Default: ''
Environment Variable: AIRFLOW__OPENSEARCH__INDEX_PATTERNS_CALLABLE
Example: module.callable

json_fields¶

Log fields to also attach to the json output, if enabled

Type: string
Default: asctime, filename, lineno, levelname, message
Environment Variable: AIRFLOW__OPENSEARCH__JSON_FIELDS

json_format¶

Instead of the default log formatter, write the log lines as JSON

Type: string
Default: False
Environment Variable: AIRFLOW__OPENSEARCH__JSON_FORMAT

log_id_template¶

Format of the log_id, which is used to query for a given tasks logs

Type: string
Default: {dag_id}-{task_id}-{run_id}-{map_index}-{try_number}
Environment Variable: AIRFLOW__OPENSEARCH__LOG_ID_TEMPLATE

offset_field¶

The field where offset is stored (normally either offset or log.offset)

Type: string
Default: offset
Environment Variable: AIRFLOW__OPENSEARCH__OFFSET_FIELD

password¶

The password for connecting to Opensearch

Type

string

Default

''

Environment Variables

AIRFLOW__OPENSEARCH__PASSWORD

AIRFLOW__OPENSEARCH__PASSWORD_CMD

AIRFLOW__OPENSEARCH__PASSWORD_SECRET

port¶

The port number of Opensearch host

Type: integer
Default: ''
Environment Variable: AIRFLOW__OPENSEARCH__PORT

username¶

The username for connecting to Opensearch

Type

string

Default

''

Environment Variables

AIRFLOW__OPENSEARCH__USERNAME

AIRFLOW__OPENSEARCH__USERNAME_CMD

AIRFLOW__OPENSEARCH__USERNAME_SECRET

write_stdout¶

Write the task logs to the stdout of the worker, rather than the default files

Type: string
Default: False
Environment Variable: AIRFLOW__OPENSEARCH__WRITE_STDOUT

[opensearch_configs]¶

ca_certs¶

Type: string
Default: ''
Environment Variable: AIRFLOW__OPENSEARCH_CONFIGS__CA_CERTS

http_compress¶

Type: string
Default: False
Environment Variable: AIRFLOW__OPENSEARCH_CONFIGS__HTTP_COMPRESS

ssl_assert_hostname¶

Type: string
Default: False
Environment Variable: AIRFLOW__OPENSEARCH_CONFIGS__SSL_ASSERT_HOSTNAME

ssl_show_warn¶

Type: string
Default: False
Environment Variable: AIRFLOW__OPENSEARCH_CONFIGS__SSL_SHOW_WARN

use_ssl¶

Type: string
Default: False
Environment Variable: AIRFLOW__OPENSEARCH_CONFIGS__USE_SSL

verify_certs¶

Type: string
Default: False
Environment Variable: AIRFLOW__OPENSEARCH_CONFIGS__VERIFY_CERTS