Render map_index_template during task running state - #57208
Merged
Conversation
This was referenced Oct 24, 2025
kaxil
force-pushed
the
render-template
branch
6 times, most recently
from
October 25, 2025 23:29
3f78d7e to
ad77ee6
Compare
kaxil
marked this pull request as ready for review
October 26, 2025 00:39
Currently, custom labels defined via `map_index_template` for mapped tasks
are only rendered and displayed in the UI after task completion. This means
users see only numeric indices (0, 1, 2...) while tasks are running, making
it difficult to identify which specific mapped task instance is executing.
For example, if a mapped task processes different customers, users want to
see "customer_alice" or "customer_bob" in the UI while the task is running,
not just "0" or "1".
The `map_index_template` may reference context
variables set during task execution (e.g., XCom values, return values from
`execute()`, etc.). These variables are only available after the task's
execute method completes, which is why rendering was originally delayed
until task completion.
Example problematic template:
```
"Processing {{ ti.xcom_pull(key='customer_name') }}"
```
or one that access Task Attributes:
```
"Processing {{ task.value }}"
```
This PR implements a two-phase rendering approach:
Phase 1:
- Renders `map_index_template` immediately after template rendering, before
task execution
- Uses available context variables (task, dag, run_id, logical_date, etc.)
- Sends the rendered label to the API server for immediate UI display
- If rendering fails (e.g., due to missing variables), error is suppressed
and logged at debug level
- Provides immediate feedback for templates using pre-execution context
Phase 2:
- Re-renders `map_index_template` with full context after task execution
- Includes all execution-time variables (XCom, Task attributes, return values, etc.)
- Only sends update if the value changed from Phase 1
- Handles both success and failure cases
- For failures, suppresses rendering errors to avoid masking task errors
For templates with execution-time variables, the label updates after
execution completes with the final rendered value.
kaxil
force-pushed
the
render-template
branch
from
October 26, 2025 13:24
ad77ee6 to
e036df7
Compare
Member
|
may be we need to exclude this new type from the trigger supervisor type tests? |
eladkal
approved these changes
Oct 26, 2025
Member
Author
Yeah, excluded |
2 tasks
|
Hi, is this working with latest version 3.2.0? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Currently, custom labels defined via
map_index_templatefor mapped tasks are only rendered and displayed in the UI after task completion. This means users see only numeric indices (0, 1, 2...) while tasks are running, making it difficult to identify which specific mapped task instance is executing.For example, if a mapped task processes different customers, users want to see "customer_alice" or "customer_bob" in the UI while the task is running, not just "0" or "1".
The
map_index_templatemay reference contextvariables set during task execution (e.g., XCom values, return values from
execute(), etc.). These variables are only available after the task's execute method completes, which is why rendering was originally delayed until task completion.Example problematic template:
or one that access Task Attributes:
This PR implements a two-phase rendering approach:
Phase 1:
map_index_templateimmediately after template rendering, before task executionPhase 2:
map_index_templatewith full context after task executionFor templates with execution-time variables, the label updates after execution completes with the final rendered value.
Fixes #39118
Fixes #56722
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.