Error Tracking

Airflow can be set up to send errors to Sentry.

Setup

First you must install sentry requirement:

pip install 'apache-airflow[sentry]'

After that, you need to enable the integration by set sentry_on option in [sentry] section to "True".

Add your SENTRY_DSN to your configuration file e.g. airflow.cfg in [sentry] section. Its template resembles the following: '{PROTOCOL}://{PUBLIC_KEY}@{HOST}/{PROJECT_ID}'

[sentry]
sentry_dsn = http://foo@sentry.io/123

Note

If this value is not provided, the SDK will try to read it from the SENTRY_DSN environment variable.

You can supply additional configuration options based on the Python platform via [sentry] section. Unsupported options: integrations, in_app_include, in_app_exclude, ignore_errors, before_breadcrumb, before_send, transport.

Tags

Name

Description

dag_id

Dag name of the dag that failed

task_id

Task name of the task that failed

data_interval_start

Start of data interval when the task failed

data_interval_end

End of data interval when the task failed

operator

Operator name of the task that failed

For backward compatibility, an additional tag execution_date is also available to represent the logical date. The tag should be considered deprecated in favor of data_interval_start.

Impact of Sentry on Environment variables passed to Subprocess Hook

When Sentry is enabled, by default it changes standard library to pass all environment variables to subprocesses opened by Airflow. This changes the default behaviour of airflow.hooks.subprocess.SubprocessHook - always all environment variables are passed to the subprocess executed with specific set of environment variables. In this case not only the specified environment variables are passed but also all existing environment variables are passed with SUBPROCESS_ prefix added. This happens also for all other subprocesses.

This behaviour can be disabled by setting default_integrations sentry configuration parameter to False which disables StdlibIntegration. This however also disables other default integrations and you need to enable them manually if you want to get them enabled, see Sentry Default Integrations

[sentry]
default_integrations = False

Was this entry helpful?