Params

Params are how Airflow provides runtime configuration to tasks. When you trigger a DAG manually, you can modify its Params before the dagrun starts. If the user-supplied values don't pass validation, Airflow shows a warning instead of creating the dagrun. (For scheduled runs, the default values are used.)

Adding Params to a DAG

To add Params to a DAG, initialize it with the params kwarg. Use a dictionary that maps Param names to a either a Param or an object indicating the parameter's default value.

from airflow import DAG
from airflow.models.param import Param

with DAG(
    "the_dag",
    params={
        "x": Param(5, type="integer", minimum=3),
        "y": 6
    },
) as the_dag:

Referencing Params in a Task

Params are stored as params in the template context. So you can reference them in a template.

PythonOperator(
    task_id="from_template",
    op_args=[
        "{{ params.int_param + 10 }}",
    ],
    python_callable=(
        lambda x: print(x)
    ),
)

Even though Params can use a variety of types, the default behavior of templates is to provide your task with a string. You can change this by setting render_template_as_native_obj=True while initializing the DAG.

with DAG(
    "the_dag",
    params={"x": Param(5, type="integer", minimum=3)},
    render_template_as_native_obj=True
) as the_dag:

This way, the Param's type is respected when its provided to your task.

# prints <class 'str'> by default
# prints <class 'int'> if render_template_as_native_obj=True
PythonOperator(
    task_id="template_type",
    op_args=[
        "{{ params.int_param }}",
    ],
    python_callable=(
        lambda x: print(type(x))
    ),
)

Another way to access your param is via a task's context kwarg.

def print_x(**context):
    print(context["params"]["x"])

PythonOperator(
    task_id="print_x",
    python_callable=print_it,
)

Task-level Params

You can also add Params to individual tasks.

PythonOperator(
    task_id="print_x",
    params={"x": 10},
    python_callable=print_it,
)

If there's already a dag param with that name, the task-level default will take precedence over the dag-level default. If a user supplies their own value when the DAG was triggered, Airflow ignores all defaults and uses the user's value.

JSON Schema Validation

Param makes use of json-schema <https://json-schema.org/>, so you can use the full json-schema specifications mentioned at https://json-schema.org/draft/2020-12/json-schema-validation.html to define Param objects.

with DAG(
    "my_dag",
    params={
        # a int with a default value
        "int_param": Param(10, type="integer", minimum=0, maximum=20),

        # a required param which can be of multiple types
        "dummy": Param(type=["null", "number", "string"]),

        # an enum param, must be one of three values
        "enum_param": Param("foo", enum=["foo", "bar", 42]),

        # a param which uses json-schema formatting
        "email": Param(
            default="example@example.com",
            type="string",
            format="idn-email",
            minLength=5,
            maxLength=255,
        ),
    },
) as my_dag:

Note

As of now, for security reasons, one can not use Param objects derived out of custom classes. We are planning to have a registration system for custom Param classes, just like we've for Operator ExtraLinks.

Disabling Runtime Param Modification

The ability to update params while triggering a DAG depends on the flag core.dag_run_conf_overrides_params. Setting this config to False will effectively turn your default params into constants.

Was this entry helpful?