airflow.models.abstractoperator

Module Contents

Classes

AbstractOperator

Common implementation for operators, including unmapped and mapped.

Attributes

TaskStateChangeCallback

DEFAULT_OWNER

DEFAULT_POOL_SLOTS

DEFAULT_PRIORITY_WEIGHT

DEFAULT_QUEUE

DEFAULT_IGNORE_FIRST_DEPENDS_ON_PAST

DEFAULT_RETRIES

DEFAULT_RETRY_DELAY

DEFAULT_WEIGHT_RULE

DEFAULT_TRIGGER_RULE

DEFAULT_TASK_EXECUTION_TIMEOUT

airflow.models.abstractoperator.TaskStateChangeCallback[source]
airflow.models.abstractoperator.DEFAULT_OWNER: str[source]
airflow.models.abstractoperator.DEFAULT_POOL_SLOTS: int = 1[source]
airflow.models.abstractoperator.DEFAULT_PRIORITY_WEIGHT: int = 1[source]
airflow.models.abstractoperator.DEFAULT_QUEUE: str[source]
airflow.models.abstractoperator.DEFAULT_IGNORE_FIRST_DEPENDS_ON_PAST: bool[source]
airflow.models.abstractoperator.DEFAULT_RETRIES: int[source]
airflow.models.abstractoperator.DEFAULT_RETRY_DELAY: datetime.timedelta[source]
airflow.models.abstractoperator.DEFAULT_WEIGHT_RULE: airflow.utils.weight_rule.WeightRule[source]
airflow.models.abstractoperator.DEFAULT_TRIGGER_RULE: airflow.utils.trigger_rule.TriggerRule[source]
airflow.models.abstractoperator.DEFAULT_TASK_EXECUTION_TIMEOUT: datetime.timedelta | None[source]
exception airflow.models.abstractoperator.NotMapped[source]

Bases: Exception

Raise if a task is neither mapped nor has any parent mapped groups.

class airflow.models.abstractoperator.AbstractOperator(context=None)[source]

Bases: airflow.utils.log.logging_mixin.LoggingMixin, airflow.models.taskmixin.DAGNode

Common implementation for operators, including unmapped and mapped.

This base class is more about sharing implementations, not defining a common interface. Unfortunately it’s difficult to use this as the common base class for typing due to BaseOperator carrying too much historical baggage.

The union type from airflow.models.operator import Operator is easier to use for typing purposes.

abstract property task_type: str[source]
abstract property operator_name: str[source]
abstract property inherits_from_empty_operator: bool[source]
property dag_id: str[source]

Returns dag id if it has one or an adhoc + owner

property node_id: str[source]
property priority_weight_total: int[source]

Total priority weight for the task. It might include all upstream or downstream tasks.

Depending on the weight rule:

  • WeightRule.ABSOLUTE - only own weight

  • WeightRule.DOWNSTREAM - adds priority weight of all downstream tasks

  • WeightRule.UPSTREAM - adds priority weight of all upstream tasks

operator_class: type[BaseOperator] | dict[str, Any][source]
weight_rule: str[source]
priority_weight: int[source]
template_fields: Collection[str][source]
template_ext: Sequence[str][source]
owner: str[source]
task_id: str[source]
outlets: list[source]
inlets: list[source]
HIDE_ATTRS_FROM_UI: ClassVar[frozenset[str]][source]
abstract get_dag()[source]
get_template_env()[source]

Fetch a Jinja template environment from the DAG or instantiate empty environment if no DAG.

prepare_template()[source]

Hook triggered after the templated fields get replaced by their content.

If you need your operator to alter the content of the file before the template is rendered, it should override this method to do so.

resolve_template_files()[source]

Getting the content of files for template_field / template_ext.

get_direct_relative_ids(upstream=False)[source]

Get direct relative IDs to the current task, upstream or downstream.

get_flat_relative_ids(upstream=False, found_descendants=None)[source]

Get a flat set of relative IDs, upstream or downstream.

get_flat_relatives(upstream=False)[source]

Get a flat list of relatives, either upstream or downstream.

iter_mapped_dependants()[source]

Return mapped nodes that depend on the current task the expansion.

For now, this walks the entire DAG to find mapped nodes that has this current task as an upstream. We cannot use downstream_list since it only contains operators, not task groups. In the future, we should provide a way to record an DAG node’s all downstream nodes instead.

iter_mapped_task_groups()[source]

Return mapped task groups this task belongs to.

Groups are returned from the closest to the outmost.

get_closest_mapped_task_group()[source]
abstract unmap(resolve)[source]

Get the “normal” operator from current abstract operator.

MappedOperator uses this to unmap itself based on the map index. A non- mapped operator (i.e. BaseOperator subclass) simply returns itself.

Returns dictionary of all extra links for the operator

Returns dictionary of all global extra links

For an operator, gets the URLs that the extra_links entry points to.

Raises

ValueError – The error message of a ValueError will be passed on through to the fronted to show up as a tooltip on the disabled link.

Parameters

  • ti (airflow.models.taskinstance.TaskInstance) – The TaskInstance for the URL being searched for.

  • link_name (str) – The name of the link we’re looking for the URL for. Should be one of the options specified in extra_links.

get_parse_time_mapped_ti_count()[source]

Number of mapped task instances that can be created on DAG run creation.

This only considers literal mapped arguments, and would return None when any non-literal values are used for mapping.

Raises

  • NotFullyPopulated – If non-literal mapped arguments are encountered.

  • NotMapped – If the operator is neither mapped, nor has any parent mapped task groups.

Returns

Total number of mapped TIs this task should have.

Return type

int

get_mapped_ti_count(run_id, *, session)[source]

Number of mapped TaskInstances that can be created at run time.

This considers both literal and non-literal mapped arguments, and the result is therefore available when all depended tasks have finished. The return value should be identical to parse_time_mapped_ti_count if all mapped arguments are literal.

Raises

  • NotFullyPopulated – If upstream tasks are not all complete yet.

  • NotMapped – If the operator is neither mapped, nor has any parent mapped task groups.

Returns

Total number of mapped TIs this task should have.

Return type

int

expand_mapped_task(run_id, *, session)[source]

Create the mapped task instances for mapped task.

Raises

NotMapped – If this task does not need expansion.

Returns

The newly created mapped task instances (if any) in ascending order by map index, and the maximum map index value.

Return type

tuple[Sequence[airflow.models.taskinstance.TaskInstance], int]

abstract render_template_fields(context, jinja_env=None)[source]

Template all attributes listed in self.template_fields.

If the operator is mapped, this should return the unmapped, fully rendered, and map-expanded operator. The mapped operator should not be modified. However, context may be modified in-place to reference the unmapped operator for template rendering.

If the operator is not mapped, this should modify the operator in-place.

render_template(content, context, jinja_env=None, seen_oids=None)[source]

Render a templated string.

If content is a collection holding multiple templated strings, strings in the collection will be templated recursively.

Parameters

  • content (Any) – Content to template. Only strings can be templated (may be inside a collection).

  • context (airflow.utils.context.Context) – Dict with values to apply on templated content

  • jinja_env (jinja2.Environment | None) – Jinja environment. Can be provided to avoid re-creating Jinja environments during recursion.

  • seen_oids (set[int] | None) – template fields already rendered (to avoid RecursionError on circular dependencies)

Returns

Templated content

Return type

Any

Was this entry helpful?