airflow.decorators.base

Module Contents

Classes

DecoratedOperator

Wraps a Python callable and captures args/kwargs when called for execution.

DecoratedMappedOperator

MappedOperator implementation for @task-decorated task function.

Task

Declaration of a @task-decorated callable for type-checking.

TaskDecorator

Type declaration for task_decorator_factory return type.

Functions

get_unique_task_id(task_id[, dag, task_group])

Generate unique task id given a DAG (or if run in a DAG context).

task_decorator_factory([python_callable, multiple_outputs])

Generate a wrapper that wraps a function into an Airflow operator.

Attributes

FParams

FReturn

OperatorSubclass

airflow.decorators.base.get_unique_task_id(task_id, dag=None, task_group=None)[source]

Generate unique task id given a DAG (or if run in a DAG context).

IDs are generated by appending a unique number to the end of the original task id.

Example:

task_id task_id__1 task_id__2 … task_id__20

class airflow.decorators.base.DecoratedOperator(*, python_callable, task_id, op_args=None, op_kwargs=None, multiple_outputs=False, kwargs_to_upstream=None, **kwargs)[source]

Bases: airflow.models.baseoperator.BaseOperator

Wraps a Python callable and captures args/kwargs when called for execution.

Parameters
  • python_callable (Callable) – A reference to an object that is callable

  • op_kwargs (Mapping[str, Any] | None) – a dictionary of keyword arguments that will get unpacked in your function (templated)

  • op_args (Collection[Any] | None) – a list of positional arguments that will get unpacked when calling your callable (templated)

  • multiple_outputs (bool) – If set to True, the decorated function’s return value will be unrolled to multiple XCom values. Dict will unroll to XCom values with its keys as XCom keys. Defaults to False.

  • kwargs_to_upstream (dict[str, Any] | None) – For certain operators, we might need to upstream certain arguments that would otherwise be absorbed by the DecoratedOperator (for example python_callable for the PythonOperator). This gives a user the option to upstream kwargs as needed.

template_fields: Sequence[str] = ('op_args', 'op_kwargs')[source]
template_fields_renderers[source]
shallow_copy_attrs: Sequence[str] = ('python_callable',)[source]
execute(context)[source]

This is the main method to derive when creating an operator. Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

get_python_source()[source]
airflow.decorators.base.FParams[source]
airflow.decorators.base.FReturn[source]
airflow.decorators.base.OperatorSubclass[source]
class airflow.decorators.base.DecoratedMappedOperator(context=None)[source]

Bases: airflow.models.mappedoperator.MappedOperator

MappedOperator implementation for @task-decorated task function.

multiple_outputs: bool[source]
python_callable: Callable[source]
op_kwargs_expand_input: airflow.models.expandinput.ExpandInput[source]
__hash__()[source]

Return hash(self).

__attrs_post_init__()[source]
class airflow.decorators.base.Task[source]

Bases: airflow.typing_compat.Protocol, Generic[FParams, FReturn]

Declaration of a @task-decorated callable for type-checking.

An instance of this type inherits the call signature of the decorated function wrapped in it (not exactly since it actually returns an XComArg, but there’s no way to express that right now), and provides two additional methods for task-mapping.

This type is implemented by _TaskDecorator at runtime.

property __wrapped__: Callable[FParams, FReturn][source]
__call__: Callable[FParams, airflow.models.xcom_arg.XComArg][source]
function: Callable[FParams, FReturn][source]
partial(**kwargs)[source]
expand(**kwargs)[source]
expand_kwargs(kwargs, *, strict=True)[source]
override(**kwargs)[source]
class airflow.decorators.base.TaskDecorator[source]

Bases: airflow.typing_compat.Protocol

Type declaration for task_decorator_factory return type.

__call__(python_callable: Callable[FParams, FReturn]) Task[FParams, FReturn][source]
__call__(*, multiple_outputs: bool | None = None, **kwargs: Any) Callable[[Callable[FParams, FReturn]], Task[FParams, FReturn]]

For the decorator factory @task() case.

override(**kwargs)[source]
airflow.decorators.base.task_decorator_factory(python_callable=None, *, multiple_outputs=None, decorated_operator_class, **kwargs)[source]

Generate a wrapper that wraps a function into an Airflow operator.

Can be reused in a single DAG.

Parameters
  • python_callable (Callable | None) – Function to decorate.

  • multiple_outputs (bool | None) – If set to True, the decorated function’s return value will be unrolled to multiple XCom values. Dict will unroll to XCom values with its keys as XCom keys. If set to False (default), only at most one XCom value is pushed.

  • decorated_operator_class (type[airflow.models.baseoperator.BaseOperator]) – The operator that executes the logic needed to run the python function in the correct environment.

Other kwargs are directly forwarded to the underlying operator class when it’s instantiated.

Was this entry helpful?