Skip to content

ChatRequest reasoning cannot express max_tokens — API accepts it, SDK silently strips it #693

Open

Description

@daveycodez

Summary

The chat-completions HTTP API accepts a per-request reasoning token budget:

{ "reasoning": { "max_tokens": 2000 } }

but the SDK cannot send it. ChatRequestReasoning (models/chatrequest.ts) only models effort and summary:

export type ChatRequestReasoning = {
    effort?: ChatRequestEffort | null | undefined;
    summary?: ChatReasoningSummaryVerbosityEnum | null | undefined;
};

and because ChatRequestReasoning$outboundSchema is a plain z.object(...), an unknown key passed by a caller (e.g. via a type cast from JS/TS) is silently stripped before the request is serialized — the request goes out with no reasoning config at all, with no error or warning. That silent-drop behavior is the sharp edge: a caller who casts around the type believes they are testing a capped configuration while actually measuring the default.

Notably the SDK already models the right shape elsewhere — ReasoningConfig (models/reasoningconfig.ts) has maxTokens with a max_tokens wire mapping — it just isn't referenced by ChatRequest.

Ask

Add maxTokens to ChatRequestReasoning (serializing to max_tokens), or reference the existing ReasoningConfig from ChatRequest, so the SDK matches the documented API surface.

Use case

Effort tiers are too coarse for cost control on budget-based thinking models (Gemini flash family, Anthropic). On a large-batch classification workload the thinking tail scales with batch size — a per-request budget like max_tokens: batchItems * K caps the tail proportionally, which no fixed effort tier can express. Today the only SDK-expressible step below our current tier is a whole effort level down, which measurably degrades task quality.

Tested on @openrouter/sdk 0.13.20.

Activity

  1. mstevens843 commented on Sep 25, 2026

    @mstevens843

    @daveycodez Confirmed on @openrouter/sdk 1.3.27 (8d95d52), Node 22, TypeScript 5.8.3.

    TypeScript rejects nested reasoning.maxTokens. Using plain JavaScript through public chat.send(), I captured the outgoing body at the injected HTTPClient fetcher: reasoning: { maxTokens: 2000 } becomes "reasoning": {}. Adding summary: "auto" preserves the summary but still drops the budget.

    Controls preserve effort, summary, and top-level maxTokens; malformed effort is rejected before transport.

    No request reached OpenRouter or a provider. This verifies SDK serialization, not provider budget enforcement.

    ChatRequestReasoning$outboundSchema lists only effort and summary. Could the source schema gain max_tokens and the SDK be regenerated with the corresponding maxTokens mapping?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions