airflow.models.mappedoperator

Module Contents

Classes

OperatorPartial

An "intermediate state" returned by BaseOperator.partial().

MappedOperator

Object representing a mapped operator in a DAG.

Functions

validate_mapping_kwargs(op, func, value)

prevent_duplicates(kwargs1, kwargs2, *, fail_reason)

ensure_xcomarg_return_value(arg)

Attributes

ValidationSource

airflow.models.mappedoperator.ValidationSource[source]
airflow.models.mappedoperator.validate_mapping_kwargs(op, func, value)[source]
airflow.models.mappedoperator.prevent_duplicates(kwargs1, kwargs2, *, fail_reason)[source]
airflow.models.mappedoperator.ensure_xcomarg_return_value(arg)[source]
class airflow.models.mappedoperator.OperatorPartial[source]

An “intermediate state” returned by BaseOperator.partial().

This only exists at DAG-parsing time; the only intended usage is for the user to call .expand() on it at some point (usually in a method chain) to create a MappedOperator to add into the DAG.

operator_class :type[airflow.models.baseoperator.BaseOperator][source]
kwargs :dict[str, Any][source]
__attrs_post_init__()[source]
__repr__()[source]

Return repr(self).

__del__()[source]
expand(**mapped_kwargs)[source]
expand_kwargs(kwargs, *, strict=True)[source]
class airflow.models.mappedoperator.MappedOperator(context=None)[source]

Bases: airflow.models.abstractoperator.AbstractOperator

Object representing a mapped operator in a DAG.

operator_class :type[BaseOperator] | dict[str, Any][source]
expand_input :airflow.models.expandinput.ExpandInput[source]
partial_kwargs :dict[str, Any][source]
task_id :str[source]
params :dict | None[source]
deps :frozenset[airflow.ti_deps.deps.base_ti_dep.BaseTIDep][source]
template_ext :Sequence[str][source]
template_fields :Collection[str][source]
template_fields_renderers :dict[str, str][source]
ui_color :str[source]
ui_fgcolor :str[source]
dag :DAG | None[source]
task_group :TaskGroup | None[source]
start_date :pendulum.DateTime | None[source]
end_date :pendulum.DateTime | None[source]
upstream_task_ids :set[str][source]
downstream_task_ids :set[str][source]
is_mapped :ClassVar[bool] = True[source]
subdag :None[source]
HIDE_ATTRS_FROM_UI :ClassVar[frozenset[str]][source]
__hash__()[source]

Return hash(self).

__repr__()[source]

Return repr(self).

__attrs_post_init__()[source]
classmethod get_serialized_fields()[source]
static deps_for(operator_class)[source]
property task_type[source]

Implementing Operator.

property operator_name[source]
property inherits_from_empty_operator[source]

Implementing Operator.

property roots[source]

Implementing DAGNode.

property leaves[source]

Implementing DAGNode.

property owner[source]
property email[source]
property trigger_rule[source]
property depends_on_past[source]
property ignore_first_depends_on_past[source]
property wait_for_downstream[source]
property retries[source]
property queue[source]
property pool[source]
property pool_slots[source]
property execution_timeout[source]
property max_retry_delay[source]
property retry_delay[source]
property retry_exponential_backoff[source]
property priority_weight[source]
property weight_rule[source]
property sla[source]
property max_active_tis_per_dag[source]
property resources[source]
property on_execute_callback[source]
property on_failure_callback[source]
property on_retry_callback[source]
property on_success_callback[source]
property run_as_user[source]
property executor_config[source]
property inlets[source]
property outlets[source]
property doc[source]
property doc_md[source]
property doc_json[source]
property doc_yaml[source]
property doc_rst[source]
get_dag()[source]

Implementing Operator.

property output[source]

Returns reference to XCom pushed by current operator

serialize_for_task_group()[source]

Implementing DAGNode.

unmap(resolve)[source]

Get the “normal” Operator after applying the current mapping.

The resolve argument is only used if operator_class is a real class, i.e. if this operator is not serialized. If operator_class is not a class (i.e. this DAG has been deserialized), this returns a SerializedBaseOperator that “looks like” the actual unmapping result.

If resolve is a two-tuple (context, session), the information is used to resolve the mapped arguments into init arguments. If it is a mapping, no resolving happens, the mapping directly provides those init arguments resolved from mapped kwargs.

expand_mapped_task(run_id, *, session)[source]

Create the mapped task instances for mapped task.

Returns

The newly created mapped TaskInstances (if any) in ascending order by map index, and the maximum map_index.

Return type

tuple[Sequence[airflow.models.taskinstance.TaskInstance], int]

prepare_for_execution()[source]
iter_mapped_dependencies()[source]

Upstream dependencies that provide XComs used by this task for task mapping.

parse_time_mapped_ti_count()[source]

Number of mapped TaskInstances that can be created at DagRun create time.

Returns

None if non-literal mapped arg encountered, or the total number of mapped TIs this task should have.

Return type

int | None

run_time_mapped_ti_count(run_id, *, session)[source]

Number of mapped TaskInstances that can be created at run time.

Returns

None if upstream tasks are not complete yet, or the total number of mapped TIs this task should have.

Return type

int | None

render_template_fields(context, jinja_env=None)[source]

Template all attributes listed in template_fields.

If the operator is mapped, this should return the unmapped, fully rendered, and map-expanded operator. The mapped operator should not be modified. However, context will be modified in-place to reference the unmapped operator for template rendering.

If the operator is not mapped, this should modify the operator in-place.

Was this entry helpful?