fix(task-sdk): handle missing DAG in TriggerDagRunOperator gracefully - #63207
fix(task-sdk): handle missing DAG in TriggerDagRunOperator gracefully#63207YoannAbriel wants to merge 2 commits into
Conversation
7dac8a9 to
8b6e936
Compare
82c16cd to
898514d
Compare
175a64a to
9cd3765
Compare
9cd3765 to
9ced47b
Compare
|
|
||
| log = mock.MagicMock() | ||
| mock_supervisor_comms.send.return_value = ErrorResponse(error=ErrorType.DAG_NOT_FOUND) | ||
| state, msg, _ = run(ti, ti.get_template_context(), log) |
There was a problem hiding this comment.
New test introduces an unspecced mock.MagicMock() for log. Airflow’s testing guidelines prefer mocks with spec/autospec to avoid silently accepting unexpected attributes; consider using a real structlog.get_logger(...) or mock.Mock(spec=...) for the logger instead.
| log.error("Dag not found.", dag_id=dag_id) | ||
| return ErrorResponse(error=ErrorType.DAG_NOT_FOUND) |
There was a problem hiding this comment.
The new 404 handling logs only a generic "Dag not found" message and doesn’t include available response context (e.detail, status_code). Consider logging those fields (and/or returning them in ErrorResponse.detail) for easier debugging, consistent with other client methods’ 404 handling.
| log.error("Dag not found.", dag_id=dag_id) | |
| return ErrorResponse(error=ErrorType.DAG_NOT_FOUND) | |
| log.error( | |
| "Dag not found.", | |
| dag_id=dag_id, | |
| status_code=e.response.status_code, | |
| detail=e.detail, | |
| ) | |
| return ErrorResponse(error=ErrorType.DAG_NOT_FOUND, detail=e.detail) |
69533e8 to
e8284fe
Compare
When TriggerDagRunOperator targets a non-existent DAG, the API server returns 404 which was re-raised as ServerResponseError. This propagated out of _handle_trigger_dag_run into the finally block of run() where 'state' was never assigned, causing UnboundLocalError. Two fixes: - Initialize state to FAILED before the try block so the finally block always has a valid value - Handle 404 in the dag_runs client (returning DAG_NOT_FOUND error) and in _handle_trigger_dag_run (failing the task gracefully) Closes: apache#63089
e8284fe to
aa9068e
Compare
|
@YoannAbriel — Removing the The label's contract is that the PR is ready for maintainer review — a regression like this means the PR temporarily isn't. Rebase your branch onto the latest
No rush. Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you. |
|
@YoannAbriel This draft PR has had no activity for 2 weeks. Closing to keep the queue clean. You are welcome to reopen and continue when you're ready. If you'd like to pick it back up, please rebase onto the current Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you. |
Problem
TriggerDagRunOperatorcrashes withUnboundLocalError: cannot access local variable 'state'when targeting a non-existent DAG. The API server returns 404, whichSUPERVISOR_COMMS.send()re-raises asServerResponseError— but this propagates past all exception handlers inrun()into thefinallyblock wherestatewas declared but never assigned.Root Cause
Two issues:
state: TaskInstanceStateinrun()is declared without a default value. Any unhandled exception reaching thefinallyblock causesUnboundLocalErrorwhen accessingstate.valuefor stats.The
DagRunOperations.trigger()client method only handles 409 (conflict) — a 404 from the API server is re-raised asServerResponseError, which isn't caught by_handle_trigger_dag_run.Fix
state = TaskInstanceState.FAILEDas a safe default so thefinallyblock always works, regardless of what exception propagates.DAG_NOT_FOUNDtoErrorTypeenum.DagRunOperations.trigger(), returningErrorResponse(error=ErrorType.DAG_NOT_FOUND).DAG_NOT_FOUNDin_handle_trigger_dag_run, failing the task gracefully instead of crashing.Tests added for both the client-side 404 handling and the task runner DAG-not-found path.
Closes: #63089
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.