airflow.models.abstractoperator
¶
Module Contents¶
Classes¶
Common implementation for operators, including unmapped and mapped. |
Attributes¶
- airflow.models.abstractoperator.DEFAULT_RETRY_DELAY: datetime.timedelta[source]¶
- airflow.models.abstractoperator.DEFAULT_TRIGGER_RULE: airflow.utils.trigger_rule.TriggerRule[source]¶
- airflow.models.abstractoperator.DEFAULT_TASK_EXECUTION_TIMEOUT: datetime.timedelta | None[source]¶
- exception airflow.models.abstractoperator.NotMapped[source]¶
Bases:
Exception
Raise if a task is neither mapped nor has any parent mapped groups.
- class airflow.models.abstractoperator.AbstractOperator(context=None)[source]¶
Bases:
airflow.utils.log.logging_mixin.LoggingMixin
,airflow.models.taskmixin.DAGNode
Common implementation for operators, including unmapped and mapped.
This base class is more about sharing implementations, not defining a common interface. Unfortunately it's difficult to use this as the common base class for typing due to BaseOperator carrying too much historical baggage.
The union type
from airflow.models.operator import Operator
is easier to use for typing purposes.- property priority_weight_total: int[source]¶
Total priority weight for the task. It might include all upstream or downstream tasks.
Depending on the weight rule:
WeightRule.ABSOLUTE - only own weight
WeightRule.DOWNSTREAM - adds priority weight of all downstream tasks
WeightRule.UPSTREAM - adds priority weight of all upstream tasks
- operator_extra_links: Collection[airflow.models.baseoperator.BaseOperatorLink][source]¶
- get_template_env()[source]¶
Fetch a Jinja template environment from the DAG or instantiate empty environment if no DAG.
- prepare_template()[source]¶
Hook triggered after the templated fields get replaced by their content.
If you need your operator to alter the content of the file before the template is rendered, it should override this method to do so.
- get_direct_relative_ids(upstream=False)[source]¶
Get direct relative IDs to the current task, upstream or downstream.
- get_flat_relative_ids(upstream=False, found_descendants=None)[source]¶
Get a flat set of relative IDs, upstream or downstream.
- get_flat_relatives(upstream=False)[source]¶
Get a flat list of relatives, either upstream or downstream.
- iter_mapped_dependants()[source]¶
Return mapped nodes that depend on the current task the expansion.
For now, this walks the entire DAG to find mapped nodes that has this current task as an upstream. We cannot use
downstream_list
since it only contains operators, not task groups. In the future, we should provide a way to record an DAG node's all downstream nodes instead.
- iter_mapped_task_groups()[source]¶
Return mapped task groups this task belongs to.
Groups are returned from the closest to the outmost.
- abstract unmap(resolve)[source]¶
Get the "normal" operator from current abstract operator.
MappedOperator uses this to unmap itself based on the map index. A non- mapped operator (i.e. BaseOperator subclass) simply returns itself.
- get_extra_links(ti, link_name)[source]¶
For an operator, gets the URLs that the
extra_links
entry points to.- Raises
ValueError -- The error message of a ValueError will be passed on through to the fronted to show up as a tooltip on the disabled link.
- Parameters
ti (airflow.models.taskinstance.TaskInstance) -- The TaskInstance for the URL being searched for.
link_name (str) -- The name of the link we're looking for the URL for. Should be one of the options specified in
extra_links
.
- get_parse_time_mapped_ti_count()[source]¶
Number of mapped task instances that can be created on DAG run creation.
This only considers literal mapped arguments, and would return None when any non-literal values are used for mapping.
- Raises
NotFullyPopulated -- If non-literal mapped arguments are encountered.
NotMapped -- If the operator is neither mapped, nor has any parent mapped task groups.
- Returns
Total number of mapped TIs this task should have.
- Return type
- get_mapped_ti_count(run_id, *, session)[source]¶
Number of mapped TaskInstances that can be created at run time.
This considers both literal and non-literal mapped arguments, and the result is therefore available when all depended tasks have finished. The return value should be identical to
parse_time_mapped_ti_count
if all mapped arguments are literal.- Raises
NotFullyPopulated -- If upstream tasks are not all complete yet.
NotMapped -- If the operator is neither mapped, nor has any parent mapped task groups.
- Returns
Total number of mapped TIs this task should have.
- Return type
- expand_mapped_task(run_id, *, session)[source]¶
Create the mapped task instances for mapped task.
- Raises
NotMapped -- If this task does not need expansion.
- Returns
The newly created mapped task instances (if any) in ascending order by map index, and the maximum map index value.
- Return type
tuple[Sequence[airflow.models.taskinstance.TaskInstance], int]
- abstract render_template_fields(context, jinja_env=None)[source]¶
Template all attributes listed in self.template_fields.
If the operator is mapped, this should return the unmapped, fully rendered, and map-expanded operator. The mapped operator should not be modified. However, context may be modified in-place to reference the unmapped operator for template rendering.
If the operator is not mapped, this should modify the operator in-place.
- render_template(content, context, jinja_env=None, seen_oids=None)[source]¶
Render a templated string.
If content is a collection holding multiple templated strings, strings in the collection will be templated recursively.
- Parameters
content (Any) -- Content to template. Only strings can be templated (may be inside a collection).
context (airflow.utils.context.Context) -- Dict with values to apply on templated content
jinja_env (jinja2.Environment | None) -- Jinja environment. Can be provided to avoid re-creating Jinja environments during recursion.
seen_oids (set[int] | None) -- template fields already rendered (to avoid RecursionError on circular dependencies)
- Returns
Templated content
- Return type
Any