Part of epic #8286 (Phase 3 — AI observability).
Problem
On the failure path, $ai_model reports a model that never ran.
resolveModel (src/selfhost/ai.ts:141) exists precisely because the core passes a Workers-AI model id that is meaningless off-Workers — it discards any @cf/-prefixed value and substitutes the provider-specific self-host model.
runProviderWithOtel then captures the two paths differently:
- Success (
src/selfhost/ai.ts:1506): model: usage?.model ?? (model || "default") — usage.model is the real resolved model. Correct.
- Failure (
src/selfhost/ai.ts:1525): model: model || "default" — there is no usage when the call failed, so this reports the raw passed-in value, which is exactly the @cf/ placeholder resolveModel threw away.
Impact, measured
On the live project, every one of the 33 failed AI events is mislabelled:
Reported $ai_model |
Reported $ai_provider |
Calls |
Errors |
@cf/openai/gpt-oss-120b |
claude-code |
11 |
11 |
default |
claude-code |
12 |
12 |
@cf/baai/bge-m3 |
ai_embed |
8 |
8 |
visual-vision |
ai_vision |
2 |
2 |
Every successful call is correctly labelled (claude-sonnet-5, qwen3:8b, bge-m3:latest, qwen3-vl:8b-instruct). The result is a perfect false signal: a breakdown by model shows a set of Cloudflare model ids with a 100% error rate that this deployment does not use and has never called, while the real failing models — self-hosted ollama and the claude-code/codex CLIs — show a 0% error rate because their failures were attributed elsewhere.
The underlying failures are real and correctly captured in $ai_error: claude_code_error_429 (15), claude_stalled_no_output — no stdout within firstOutputTimeoutMs (8), ai_embed_http_400: the input length exceeds the context length (8).
This is the same class of bug the codebase already fixed once in a different sink — src/services/ai-review.ts:3525: "self-host claude-code reviews were mis-logged as the Workers-AI model ids (@cf/openai/gpt-oss-120b+...), which hid outages."
Deliverables
Related
The same fallback shape appears in withAiGenerationCapture (src/selfhost/ai.ts:1693), which additionally passes no repo context at all — worth checking in the same pass.
Part of epic #8286 (Phase 3 — AI observability).
Problem
On the failure path,
$ai_modelreports a model that never ran.resolveModel(src/selfhost/ai.ts:141) exists precisely because the core passes a Workers-AI model id that is meaningless off-Workers — it discards any@cf/-prefixed value and substitutes the provider-specific self-host model.runProviderWithOtelthen captures the two paths differently:src/selfhost/ai.ts:1506):model: usage?.model ?? (model || "default")—usage.modelis the real resolved model. Correct.src/selfhost/ai.ts:1525):model: model || "default"— there is nousagewhen the call failed, so this reports the raw passed-in value, which is exactly the@cf/placeholderresolveModelthrew away.Impact, measured
On the live project, every one of the 33 failed AI events is mislabelled:
$ai_model$ai_provider@cf/openai/gpt-oss-120bclaude-codedefaultclaude-code@cf/baai/bge-m3ai_embedvisual-visionai_visionEvery successful call is correctly labelled (
claude-sonnet-5,qwen3:8b,bge-m3:latest,qwen3-vl:8b-instruct). The result is a perfect false signal: a breakdown by model shows a set of Cloudflare model ids with a 100% error rate that this deployment does not use and has never called, while the real failing models — self-hosted ollama and the claude-code/codex CLIs — show a 0% error rate because their failures were attributed elsewhere.The underlying failures are real and correctly captured in
$ai_error:claude_code_error_429(15),claude_stalled_no_output — no stdout within firstOutputTimeoutMs(8),ai_embed_http_400: the input length exceeds the context length(8).This is the same class of bug the codebase already fixed once in a different sink —
src/services/ai-review.ts:3525: "self-host claude-code reviews were mis-logged as the Workers-AI model ids (@cf/openai/gpt-oss-120b+...), which hid outages."Deliverables
resolveModel's output is what the provider used; the rawmodelargument is a request-layer placeholder and should never reach telemetry.@cf/-prefixed value as$ai_modelfrom the self-host path — Workers AI has no live binding anywhere (src/services/ai-summaries.ts:32), so any such value in telemetry is by definition wrong."default"is equally unhelpful as a model label; prefer the provider's own resolved default.Related
The same fallback shape appears in
withAiGenerationCapture(src/selfhost/ai.ts:1693), which additionally passes no repo context at all — worth checking in the same pass.