Preserve structured output stop reasons

[sylvesterkaczmarek]

Summary

Preserve structured-output responses when generation terminates with stop_reason="refusal" or "max_tokens" instead of replacing the response with a schema-validation exception.

messages.parse() and the beta equivalent currently attempt to validate every text block against output_format unconditionally. Refusal text and max-token-truncated JSON are not guaranteed to satisfy the requested output schema, so those terminal responses can fail inside Pydantic/JSON parsing before the caller can inspect the model's actual stop_reason.

For example, a refusal containing ordinary refusal text is currently fed to TypeAdapter.validate_json(), and a response truncated at {"value": is treated as malformed structured output. In both cases the SDK loses the more important information: why generation stopped.

Fix

Skip structured-output validation only for:

• refusal
• max_tokens

The text remains available unchanged, parsed_output is None, and the original stop_reason is preserved.

Normal completed responses continue through the existing validation path, so malformed structured output on an ordinary completed turn still raises as before.

The behavior is applied symmetrically to GA and beta parsed messages.

Regression coverage

Adds tests verifying:

• refusal text returns with parsed_output=None for GA and beta;
• truncated JSON from max_tokens returns with parsed_output=None;
• the original stop reason is preserved;
• invalid structured output on a normally completed response still raises a validation error.

[HardMax71]

@sylvesterkaczmarek the streaming path has the same problem, and this PR doesn't touch it. The stream accumulator parses at content_block_stop (_messages.py#L510-L513, same in _beta_messages.py#L544-L547), but stop_reason only arrives with the following message_delta. So client.messages.stream(..., output_format=Model) raises a raw pydantic.ValidationError mid-stream when the reply is cut at max_tokens, and the caller never gets to see the stop reason.

Offline repro (stub transport, the reply is cut at {"lane": "A-B", "pri with stop_reason: max_tokens):

repro

# Offline: messages.stream(output_format=Model) where the reply is cut off at
# max_tokens. Does the caller get stop_reason="max_tokens", or a raw pydantic
# ValidationError before stop_reason is known? Stub transport, synthetic SSE.
import json

import anthropic
try:
    import httpx2 as httpx  # anthropic>=1.0 ships on httpx2
except ImportError:
    import httpx
import pydantic
from pydantic import BaseModel


class Rate(BaseModel):
    lane: str
    price: float


def sse(events):
    out = []
    for ev in events:
        out.append(f"event: {ev['type']}\ndata: {json.dumps(ev)}\n\n")
    return "".join(out).encode()


EVENTS = [
    {"type": "message_start", "message": {"id": "msg_1", "type": "message", "role": "assistant",
        "model": "claude-test", "content": [], "stop_reason": None, "stop_sequence": None,
        "usage": {"input_tokens": 10, "output_tokens": 1}}},
    {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}},
    {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": '{"lane": "A-B", "pri'}},
    {"type": "content_block_stop", "index": 0},
    {"type": "message_delta", "delta": {"stop_reason": "max_tokens", "stop_sequence": None},
        "usage": {"output_tokens": 8}},
    {"type": "message_stop"},
]


def handler(request: httpx.Request) -> httpx.Response:
    return httpx.Response(200, headers={"content-type": "text/event-stream"}, content=sse(EVENTS))


client = anthropic.Anthropic(api_key="sk-test", http_client=httpx.Client(transport=httpx.MockTransport(handler)), max_retries=0)
print("anthropic", anthropic.__version__)
try:
    with client.messages.stream(model="claude-test", max_tokens=8, output_format=Rate,
                                messages=[{"role": "user", "content": "x"}]) as stream:
        msg = stream.get_final_message()
    print("stop_reason:", msg.stop_reason, "parsed_output:", msg.parsed_output)
except pydantic.ValidationError as e:
    print("pydantic.ValidationError raised mid-stream, stop_reason never reached:", e.errors()[0]["type"])

anthropic 1.8.0
pydantic.ValidationError raised mid-stream, stop_reason never reached: json_invalid

Same on 0.87.0. Could this PR move that parse to message_stop, where stop_reason is known, with the same refusal / max_tokens check? Or I can open a separate PR for the stream side if you'd rather keep this one small.

[sylvesterkaczmarek]

@HardMax71 Confirmed. The streaming path had the same event-ordering issue: content_block_stop runs before message_delta, so structured-output validation could fail before stop_reason was available.

Updated in signed commit 32d3d1f:

• successful structured output can still parse when the content block closes;
• validation failures are deferred until message_stop, after the stop reason is known;
• refusal and max_tokens now preserve the text and return parsed_output=None instead of masking the terminal reason;
• invalid structured output on a normal completed response still raises;
• the behavior is covered in both GA and beta streaming paths.

Focused validation: 66 streaming/parsing tests passed, including public-stream regressions for max_tokens.

[HardMax71]

@sylvesterkaczmarek I checked the current head (d1f2ed7) offline with a mock transport. A stream cut at max_tokens or ending in refusal now returns the stop reason with parsed_output=None on messages, beta.messages and the async client, valid output still parses, and invalid output on end_turn still raises. On 1.11.0 the same max_tokens and refusal streams raise ValidationError (json_invalid) before stop_reason arrives.

Could a maintainer approve the CI run and review this?

repro and output

import asyncio
import json

import anthropic
import httpx2
import pydantic
from pydantic import BaseModel


class Rate(BaseModel):
    lane: str
    price: float


def sse(text, stop_reason):
    events = [
        {"type": "message_start", "message": {"id": "msg_1", "type": "message", "role": "assistant",
            "model": "claude-test", "content": [], "stop_reason": None, "stop_sequence": None,
            "usage": {"input_tokens": 10, "output_tokens": 1}}},
        {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}},
        {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": text}},
        {"type": "content_block_stop", "index": 0},
        {"type": "message_delta", "delta": {"stop_reason": stop_reason, "stop_sequence": None},
            "usage": {"output_tokens": 8}},
        {"type": "message_stop"},
    ]
    body = "".join(f"event: {e['type']}\ndata: {json.dumps(e)}\n\n" for e in events).encode()
    return lambda request: httpx2.Response(200, headers={"content-type": "text/event-stream"}, content=body)


CASES = [
    ("cut at max_tokens", '{"lane": "A-B", "pri', "max_tokens"),
    ("refusal", "I can't help with that.", "refusal"),
    ("valid, end_turn", '{"lane": "A-B", "price": 1.5}', "end_turn"),
    ("invalid, end_turn", '{"lane": "A-B"}', "end_turn"),
]
ARGS = dict(model="claude-test", max_tokens=8, output_format=Rate, messages=[{"role": "user", "content": "x"}])


def run_sync(messages):
    with messages.stream(**ARGS) as stream:
        return stream.get_final_message()


async def run_async(messages):
    async with messages.stream(**ARGS) as stream:
        return await stream.get_final_message()


print("anthropic", anthropic.__version__)
for name, text, stop in CASES:
    for kind in ("messages", "beta.messages", "async messages"):
        transport = httpx2.MockTransport(sse(text, stop))
        try:
            if kind.startswith("async"):
                client = anthropic.AsyncAnthropic(api_key="sk-test", max_retries=0,
                                                  http_client=httpx2.AsyncClient(transport=transport))
                msg = asyncio.run(run_async(client.messages))
            else:
                client = anthropic.Anthropic(api_key="sk-test", max_retries=0,
                                             http_client=httpx2.Client(transport=transport))
                msg = run_sync(client.beta.messages if kind == "beta.messages" else client.messages)
            result = f"stop_reason={msg.stop_reason} parsed_output={msg.parsed_output!r}"
        except pydantic.ValidationError as e:
            result = f"ValidationError ({e.errors()[0]['type']})"
        print(f"{name:<18} {kind:<15} {result}")

1.11.0:

cut at max_tokens  messages        ValidationError (json_invalid)
cut at max_tokens  beta.messages   ValidationError (json_invalid)
cut at max_tokens  async messages  ValidationError (json_invalid)
refusal            messages        ValidationError (json_invalid)
refusal            beta.messages   ValidationError (json_invalid)
refusal            async messages  ValidationError (json_invalid)
valid, end_turn    messages        stop_reason=end_turn parsed_output=Rate(lane='A-B', price=1.5)
valid, end_turn    beta.messages   stop_reason=end_turn parsed_output=Rate(lane='A-B', price=1.5)
valid, end_turn    async messages  stop_reason=end_turn parsed_output=Rate(lane='A-B', price=1.5)
invalid, end_turn  messages        ValidationError (missing)
invalid, end_turn  beta.messages   ValidationError (missing)
invalid, end_turn  async messages  ValidationError (missing)

PR head d1f2ed7:

cut at max_tokens  messages        stop_reason=max_tokens parsed_output=None
cut at max_tokens  beta.messages   stop_reason=max_tokens parsed_output=None
cut at max_tokens  async messages  stop_reason=max_tokens parsed_output=None
refusal            messages        stop_reason=refusal parsed_output=None
refusal            beta.messages   stop_reason=refusal parsed_output=None
refusal            async messages  stop_reason=refusal parsed_output=None
valid, end_turn    messages        stop_reason=end_turn parsed_output=Rate(lane='A-B', price=1.5)
valid, end_turn    beta.messages   stop_reason=end_turn parsed_output=Rate(lane='A-B', price=1.5)
valid, end_turn    async messages  stop_reason=end_turn parsed_output=Rate(lane='A-B', price=1.5)
invalid, end_turn  messages        ValidationError (missing)
invalid, end_turn  beta.messages   ValidationError (missing)
invalid, end_turn  async messages  ValidationError (missing)

[sylvesterkaczmarek]

@dtmeadows-ant When convenient, could you take a look at this and approve the external-fork workflows? @HardMax71 independently reproduced the streaming failure on 1.11.0 and confirmed the current head fixes max_tokens and refusal across sync, beta, and async paths while preserving normal validation failures. The branch is current with main.