Skip to content

Fix shared AI-SDK tool-call streaming/error-handling gaps #797

Description

@edelauna

Fix shared AI-SDK tool-call streaming/error-handling gaps

Problem

src/api/transform/ai-sdk.ts's processAiSdkStreamPart and src/api/providers/openai-compatible.ts (the shared base class used by Novita, Moonshot, and Poe) have two confirmed defects, surfaced while integrating Novita AI in PR #697:

  1. Missing error handling. OpenAICompatibleHandler.createMessage/completePrompt have no try/catch, unlike 10+ sibling providers (deepseek, openai, xai, requesty, etc.) that call handleOpenAIError/handleProviderError. Because of this, thrown errors never get tagged with the provider name or a .status field, which means Task.ts's backoffAndAnnounce (error?.status === 429) never fires 429-aware retry/backoff for Novita or Moonshot, and the status-coded error UI silently degrades to a generic message. ai-sdk.ts's "error" stream-chunk case also drops statusCode/url/responseBody from the AI SDK's APICallError, keeping only .message — so even a try/catch fix wouldn't recover in-stream errors without also fixing this mapping.
  2. Tool-call dedup keyed only on toolCallId. The dedup logic added to fix Novita's tool-call streaming (seenStreamingToolCallIds / completedToolCallMarker) suppresses a second event sharing an already-seen toolCallId, without checking toolName/arguments. If a backend ever reused an id for two genuinely different tool calls, the second one is silently dropped with no error or log.

Context

This logic is shared across every OpenAICompatibleHandler-based provider (currently Moonshot and Poe, plus the new Novita), so a fix here needs to be regression-checked against all of them — not just re-validated against Novita's own e2e suite, which is how the current shape of this code was arrived at (7 same-day fixup commits, e2e-driven, with no design note).

Desired behavior

  • OpenAICompatibleHandler.createMessage/completePrompt wrap their request in a try/catch and call handleProviderError (or equivalent), so errors are provider-tagged and carry .status consistently with sibling providers.
  • ai-sdk.ts's "error" case surfaces statusCode/url/responseBody from APICallError rather than only .message.
  • Tool-call dedup keys on a composite of toolCallId and a fingerprint of (toolName, arguments), so only truly-identical repeated events are suppressed; a same-id-different-payload event is treated as distinct (or flagged) rather than silently dropped.
  • Poe's and Moonshot's existing streaming/error-handling behavior is confirmed unaffected (or intentionally improved) by these changes.

Constraints / preferences

  • This PR should land before the Novita provider-add PR, which will rebase on top of it.
  • Add test coverage for: a malformed/missing tool-call id, an empty-string tool-input-delta, a same-id-different-payload tool-call collision, and a mocked 4xx/5xx/malformed-JSON response asserting the resulting error carries the provider name and status code.

Acceptance criteria

Given a Novita or Moonshot request that receives a 429 response
When the error propagates through OpenAICompatibleHandler
Then error.status is populated and Task.ts's backoffAndAnnounce applies 429-aware retry delay
And the status-coded error UI renders correctly instead of falling back to a generic message

Given two distinct tool calls that share the same toolCallId
When processAiSdkStreamPart processes both
Then both are surfaced (or an anomaly is explicitly flagged), not silently dropped

Request checklist

  • I've searched existing Issues and Discussions for duplicates
  • This describes a specific problem with clear context and impact

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions