Fix output type loss in AgentOperator's human-in-the-loop review - #70132
Open
ColtenOuO wants to merge 1 commit into
Open
Fix output type loss in AgentOperator's human-in-the-loop review#70132ColtenOuO wants to merge 1 commit into
ColtenOuO wants to merge 1 commit into
Conversation
ColtenOuO
force-pushed
the
fix-hitl-review-output-type-loss
branch
from
July 20, 2026 14:35
dec8555 to
5c61485
Compare
kaxil
reviewed
Jul 21, 2026
kaxil
reviewed
Jul 30, 2026
ColtenOuO
force-pushed
the
fix-hitl-review-output-type-loss
branch
from
August 2, 2026 10:04
5c61485 to
a632d04
Compare
Add dump_output_to_json() to utils/output_type.py, alongside rehydrate_pydantic_output(), so both directions of the review round-trip live in one module. Route all three serialization sites through it: HITLReviewMixin._to_string, AgentOperator.regenerate_with_feedback and LLMApprovalMixin.defer_for_approval. Serialize non-str, non-BaseModel output with TypeAdapter(...).dump_json() rather than str(output), which produced a Python repr instead of JSON, and fall back to str(output) when the value's type has no pydantic schema. Split schema build from validation in rehydrate_pydantic_output(). An output_type that TypeAdapter cannot build a schema for now falls back to json.loads() instead of raising PydanticSchemaGenerationError or AttributeError, neither of which the previous except clause caught. Use rehydrate_pydantic_output() in AgentOperator.execute()'s HITL branch in place of the bespoke json.loads()/except fallback. With the default output_type=str the approved string is no longer parsed as JSON. Add tests for both schema-build guards, for dump_output_to_json(), and for output_type=str with a JSON-parseable approved string, and fold the four TestToString cases into a single delegation test.
ColtenOuO
force-pushed
the
fix-hitl-review-output-type-loss
branch
from
August 2, 2026 19:03
a632d04 to
2adcc5a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AgentOperator's human-in-the-loop review path (enable_hitl_review=True) loses the original output type for anyoutput_typethat is neitherstrnor a PydanticBaseModel(e.g.list[str],bool,dict).The output is round-tripped through a string while it's shown to a human reviewer:
HITLReviewMixin._to_stringserializes it before review, andAgentOperator.execute()deserializes it back intooutput_typeafter approval. The serialization side usedstr(output), which produces a Python repr (e.g."['tag-a', 'tag-b']"), not valid JSON.The deserialization side then tried
json.loads()on that repr, which always raised, and silently fell back to returning the raw repr string instead of the original list/bool/dict — a type change downstream tasks don't expect.regenerate_with_feedback()(used when a reviewer requests changes) had the identicalstr(output)bug.This is the same defect class already fixed for the
require_approval=Truepath in #70075, which switched toTypeAdapter(...).dump_json(...). That fix wasn't applied to the separateenable_hitl_review=Truepath, so it regressed here.Changes
utils/output_type.py: the existingrehydrate_pydantic_output()and a newdump_output_to_json().dump_output_to_json()replaces the three copies of the serialization blockthat had drifted apart —
HITLReviewMixin._to_string,AgentOperator.regenerate_with_feedbackandLLMApprovalMixin.defer_for_approval.That drift is how the original bug survived Preserve output_type through human approval in LLM operators #70075.
AgentOperator.execute()'s HITL-review branch reusesrehydrate_pydantic_outputinstead of a bespoke
json.loads/exceptfallback, so both review paths behaveconsistently.
Guarding an
output_typepydantic cannot build a schema forAgentOperatorpassesoutput_typestraight through toAgent(...), so pydantic-ai's[A, B]multi-output lists,ToolOutput/NativeOutput/PromptedOutputmarkers andoutput functions all reach these helpers.
TypeAdapterraisesPydanticSchemaGenerationError(aRuntimeError, not aValidationError) orAttributeErroron those, which neither side'sexceptclause caught:rehydrate_pydantic_outputnow separates schema build from validation and fallsback to plain
json.loads. Without this an already-approved output is lost, afterthe model call and after the reviewer has signed off.
dump_output_to_jsonfalls back tostr(output). This one fires before the sessionXCom push, so without it the task dies before the reviewer is ever shown a review
session. It is reachable through output functions, which only warn
("Falling back to unconstrained schema") for a non-schema-able return type and then
run fine.
Behaviour change
With the default
output_type=str, the approved string is no longer parsed as JSON:'42'42'42''{"text": "hi"}'{'text': 'hi'}'{"text": "hi"}''true'True'true''null'None'null'This matches the non-HITL path (which returns
result.outputunparsed) and therequire_approvalpath.stris the defaultoutput_typeandenable_hitl_reviewhas shipped since 0.1.0, so a Dag relying on the implicit parse needs an explicit
output_type(e.g.output_type=int) or its ownjson.loads()downstream.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines