Metrics

Airflow can be set up to send metrics to StatsD.

Setup

First you must install statsd requirement:

pip install 'apache-airflow[statsd]'

Note

On November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. This resolver does not yet work with Apache Airflow and might leads to errors in installation - depends on your choice of extras. In order to install Airflow you need to either downgrade pip to version 20.2.4 pip upgrade --pip==20.2.4 or, in case you use Pip 20.3, you need to add option --use-deprecated legacy-resolver to your pip install command.

Add the following lines to your configuration file e.g. airflow.cfg

[scheduler]
statsd_on = True
statsd_host = localhost
statsd_port = 8125
statsd_prefix = airflow

If you want to avoid send all the available metrics to StatsD, you can configure an allow list of prefixes to send only the metrics that start with the elements of the list:

[scheduler]
statsd_allow_list = scheduler,executor,dagrun

Counters

Name

Description

<job_name>_start

Number of started <job_name> job, ex. SchedulerJob, LocalTaskJob

<job_name>_end

Number of ended <job_name> job, ex. SchedulerJob, LocalTaskJob

operator_failures_<operator_name>

Operator <operator_name> failures

operator_successes_<operator_name>

Operator <operator_name> successes

ti_failures

Overall task instances failures

ti_successes

Overall task instances successes

zombies_killed

Zombie tasks killed

scheduler_heartbeat

Scheduler heartbeats

dag_processing.processes

Number of currently running DAG parsing processes

scheduler.tasks.killed_externally

Number of tasks killed externally

Gauges

Name

Description

dagbag_size

DAG bag size

dag_processing.import_errors

Number of errors from trying to parse DAG files

dag_processing.total_parse_time

Seconds taken to scan and import all DAG files once

dag_processing.last_runtime.<dag_file>

Seconds spent processing <dag_file> (in most recent iteration)

dag_processing.last_run.seconds_ago.<dag_file>

Seconds since <dag_file> was last processed

dag_processing.processor_timeouts

Number of file processors that have been killed due to taking too long

executor.open_slots

Number of open slots on executor

executor.queued_tasks

Number of queued tasks on executor

executor.running_tasks

Number of running tasks on executor

pool.open_slots.<pool_name>

Number of open slots in the pool

pool.used_slots.<pool_name>

Number of used slots in the pool

pool.starving_tasks.<pool_name>

Number of starving tasks in the pool

Timers

Name

Description

dagrun.dependency-check.<dag_id>

Milliseconds taken to check DAG dependencies

dag.<dag_id>.<task_id>.duration

Milliseconds taken to finish a task

dag_processing.last_duration.<dag_file>

Milliseconds taken to load the given DAG file

dagrun.duration.success.<dag_id>

Milliseconds taken for a DagRun to reach success state

dagrun.duration.failed.<dag_id>

Milliseconds taken for a DagRun to reach failed state

dagrun.schedule_delay.<dag_id>

Milliseconds of delay between the scheduled DagRun start date and the actual DagRun start date

dagrun.<dag_id>.first_task_scheduling_delay

Milliseconds elapsed between first task start_date and dagrun expected start

Was this entry helpful?