tests.system.openlineage.example_openlineage_clear_dag¶
Dags that run tasks of another Dag again by clearing them through the REST API, like a user does in the UI.
- It triggers two child Dags, waits for each to finish and clears it, one step after the other:
a Dag run with a failed task and a task that retried twice before it succeeded: only its failed tasks cleared, then the whole Dag run
a Dag run with a mapped task whose second map index retried, and a task group: one map index cleared, then the whole expansion, then the task group
It needs a deployment with an API server (the OpenLineage e2e tests).
- It checks, for every execution of every task:
taskInstance’s try_number, max_tries (which a clear raises by the retries of the task) and state
taskInstance’s map_index, rendered_map_index, expanded_ti_count and region_index of the mapped task
dagRun’s clear_number on task and Dag run events, and the Dag runId of each clear as the parent of its task events
tasks that aren’t cleared don’t run again
every execution (clear, try, map index) has its own runId, shared by its START and COMPLETE/FAIL, and every clear of the Dag run its own Dag runId (the runId check of OpenLineageTestOperator)
Attributes¶
Functions¶
|
|
|
|
|
|