airflow.models.mappedoperator

Module Contents

Classes

OperatorPartial

An "intermediate state" returned by BaseOperator.partial().

MappedOperator

Object representing a mapped operator in a DAG.

Functions

validate_mapping_kwargs(op, func, value)

ensure_xcomarg_return_value(arg)

Attributes

ValidationSource

airflow.models.mappedoperator.ValidationSource[source]
airflow.models.mappedoperator.validate_mapping_kwargs(op, func, value)[source]
airflow.models.mappedoperator.ensure_xcomarg_return_value(arg)[source]
class airflow.models.mappedoperator.OperatorPartial[source]

An "intermediate state" returned by BaseOperator.partial().

This only exists at DAG-parsing time; the only intended usage is for the user to call .expand() on it at some point (usually in a method chain) to create a MappedOperator to add into the DAG.

operator_class: type[airflow.models.baseoperator.BaseOperator][source]
kwargs: dict[str, Any][source]
params: ParamsDict | dict[source]
__attrs_post_init__()[source]
__repr__()[source]

Return repr(self).

__del__()[source]
expand(**mapped_kwargs)[source]
expand_kwargs(kwargs, *, strict=True)[source]
class airflow.models.mappedoperator.MappedOperator(context=None)[source]

Bases: airflow.models.abstractoperator.AbstractOperator

Object representing a mapped operator in a DAG.

property task_type: str[source]

Implementing Operator.

property operator_name: str[source]
property inherits_from_empty_operator: bool[source]

Implementing Operator.

property roots: Sequence[airflow.models.abstractoperator.AbstractOperator][source]

Implementing DAGNode.

property leaves: Sequence[airflow.models.abstractoperator.AbstractOperator][source]

Implementing DAGNode.

property owner: str[source]
property email: None | str | Iterable[str][source]
property trigger_rule: airflow.utils.trigger_rule.TriggerRule[source]
property depends_on_past: bool[source]
property ignore_first_depends_on_past: bool[source]
property wait_for_downstream: bool[source]
property retries: int | None[source]
property queue: str[source]
property pool: str[source]
property pool_slots: str | None[source]
property execution_timeout: datetime.timedelta | None[source]
property max_retry_delay: datetime.timedelta | None[source]
property retry_delay: datetime.timedelta[source]
property retry_exponential_backoff: bool[source]
property priority_weight: int[source]
property weight_rule: int[source]
property sla: datetime.timedelta | None[source]
property max_active_tis_per_dag: int | None[source]
property resources: Resources | None[source]
property on_execute_callback: TaskStateChangeCallback | None[source]
property on_failure_callback: TaskStateChangeCallback | None[source]
property on_retry_callback: TaskStateChangeCallback | None[source]
property on_success_callback: TaskStateChangeCallback | None[source]
property run_as_user: str | None[source]
property executor_config: dict[source]
property inlets: list[Any][source]
property outlets: list[Any][source]
property doc: str | None[source]
property doc_md: str | None[source]
property doc_json: str | None[source]
property doc_yaml: str | None[source]
property doc_rst: str | None[source]
property output: airflow.models.xcom_arg.XComArg[source]

Returns reference to XCom pushed by current operator

operator_class: type[BaseOperator] | dict[str, Any][source]
expand_input: airflow.models.expandinput.ExpandInput[source]
partial_kwargs: dict[str, Any][source]
task_id: str[source]
params: ParamsDict | dict[source]
deps: frozenset[airflow.ti_deps.deps.base_ti_dep.BaseTIDep][source]
template_ext: Sequence[str][source]
template_fields: Collection[str][source]
template_fields_renderers: dict[str, str][source]
ui_color: str[source]
ui_fgcolor: str[source]
dag: DAG | None[source]
task_group: TaskGroup | None[source]
start_date: pendulum.DateTime | None[source]
end_date: pendulum.DateTime | None[source]
upstream_task_ids: set[str][source]
downstream_task_ids: set[str][source]
subdag: None[source]
HIDE_ATTRS_FROM_UI: ClassVar[frozenset[str]][source]
__hash__()[source]

Return hash(self).

__repr__()[source]

Return repr(self).

__attrs_post_init__()[source]
classmethod get_serialized_fields()[source]
static deps_for(operator_class)[source]
get_dag()[source]

Implementing Operator.

serialize_for_task_group()[source]

Implementing DAGNode.

unmap(resolve)[source]

Get the "normal" Operator after applying the current mapping.

The resolve argument is only used if operator_class is a real class, i.e. if this operator is not serialized. If operator_class is not a class (i.e. this DAG has been deserialized), this returns a SerializedBaseOperator that "looks like" the actual unmapping result.

If resolve is a two-tuple (context, session), the information is used to resolve the mapped arguments into init arguments. If it is a mapping, no resolving happens, the mapping directly provides those init arguments resolved from mapped kwargs.

prepare_for_execution()[source]
iter_mapped_dependencies()[source]

Upstream dependencies that provide XComs used by this task for task mapping.

get_parse_time_mapped_ti_count()[source]

Number of mapped task instances that can be created on DAG run creation.

This only considers literal mapped arguments, and would return None when any non-literal values are used for mapping.

Raises

  • NotFullyPopulated -- If non-literal mapped arguments are encountered.

  • NotMapped -- If the operator is neither mapped, nor has any parent mapped task groups.

Returns

Total number of mapped TIs this task should have.

Return type

int

get_mapped_ti_count(run_id, *, session)[source]

Number of mapped TaskInstances that can be created at run time.

This considers both literal and non-literal mapped arguments, and the result is therefore available when all depended tasks have finished. The return value should be identical to parse_time_mapped_ti_count if all mapped arguments are literal.

Raises

  • NotFullyPopulated -- If upstream tasks are not all complete yet.

  • NotMapped -- If the operator is neither mapped, nor has any parent mapped task groups.

Returns

Total number of mapped TIs this task should have.

Return type

int

render_template_fields(context, jinja_env=None)[source]

Template all attributes listed in self.template_fields.

This updates context to reference the map-expanded task and relevant information, without modifying the mapped operator. The expanded task in context is then rendered in-place.

Parameters

  • context (airflow.utils.context.Context) -- Context dict with values to apply on content.

  • jinja_env (jinja2.Environment | None) -- Jinja environment to use for rendering.

Was this entry helpful?