airflow.triggers.external_task

Module Contents

Classes

WorkflowTrigger

A trigger to monitor tasks, task group and dag execution in Apache Airflow.

TaskStateTrigger

Waits asynchronously for a task in a different DAG to complete for a specific logical date.

DagStateTrigger

Waits asynchronously for a DAG to complete for a specific logical date.
class airflow.triggers.external_task.WorkflowTrigger(external_dag_id, execution_dates, external_task_ids=None, external_task_group_id=None, failed_states=None, skipped_states=None, allowed_states=None, poke_interval=2.0, soft_fail=False, **kwargs)[source]

Bases: airflow.triggers.base.BaseTrigger

A trigger to monitor tasks, task group and dag execution in Apache Airflow.

Parameters

  • external_dag_id (str) – The ID of the external DAG.

  • execution_dates (list) – A list of execution dates for the external DAG.

  • external_task_ids (Collection[str] | None) – A collection of external task IDs to wait for.

  • external_task_group_id (str | None) – The ID of the external task group to wait for.

  • failed_states (Iterable[str] | None) – States considered as failed for external tasks.

  • skipped_states (Iterable[str] | None) – States considered as skipped for external tasks.

  • allowed_states (Iterable[str] | None) – States considered as successful for external tasks.

  • poke_interval (float) – The interval (in seconds) for poking the external tasks.

  • soft_fail (bool) – If True, the trigger will not fail the entire DAG on external task failure.

serialize()[source]

Serialize the trigger param and module path.

async run()[source]

Check periodically tasks, task group or dag status.

class airflow.triggers.external_task.TaskStateTrigger(dag_id, execution_dates, trigger_start_time, states=None, task_id=None, poll_interval=2.0)[source]

Bases: airflow.triggers.base.BaseTrigger

Waits asynchronously for a task in a different DAG to complete for a specific logical date.

Parameters

  • dag_id (str) – The dag_id that contains the task you want to wait for

  • task_id (str | None) – The task_id that contains the task you want to wait for.

  • states (list[str] | None) – allowed states, default is ['success']

  • execution_dates (list[datetime.datetime]) – task execution time interval

  • poll_interval (float) – The time interval in seconds to check the state. The default value is 5 sec.

  • trigger_start_time (datetime.datetime) – time in Datetime format when the trigger was started. Is used to control the execution of trigger to prevent infinite loop in case if specified name of the dag does not exist in database. It will wait period of time equals _timeout_sec parameter from the time, when the trigger was started and if the execution lasts more time than expected, the trigger will terminate with ‘timeout’ status.

serialize()[source]

Serialize TaskStateTrigger arguments and classpath.

async run()[source]

Check periodically in the database to see if the dag exists and is in the running state.

If found, wait until the task specified will reach one of the expected states. If dag with specified name was not in the running state after _timeout_sec seconds after starting execution process of the trigger, terminate with status ‘timeout’.

count_running_dags(session)[source]

Count how many dag instances in running state in the database.

count_tasks(*, session=NEW_SESSION)[source]

Count how many task instances in the database match our criteria.

class airflow.triggers.external_task.DagStateTrigger(dag_id, states, execution_dates, poll_interval=5.0)[source]

Bases: airflow.triggers.base.BaseTrigger

Waits asynchronously for a DAG to complete for a specific logical date.

Parameters

  • dag_id (str) – The dag_id that contains the task you want to wait for

  • states (list[airflow.utils.state.DagRunState]) – allowed states, default is ['success']

  • execution_dates (list[datetime.datetime]) – The logical date at which DAG run.

  • poll_interval (float) – The time interval in seconds to check the state. The default value is 5.0 sec.

serialize()[source]

Serialize DagStateTrigger arguments and classpath.

async run()[source]

Check periodically if the dag run exists, and has hit one of the states yet, or not.

count_dags(*, session=NEW_SESSION)[source]

Count how many dag runs in the database match our criteria.

Was this entry helpful?