A refused round explained itself with an HTTP header dump (#476, code half) by rbuergi · Pull Request #961 · Systemorph/MeshWeaver

rbuergi · 2026-08-08T20:30:10Z

Completes the last open piece of the code half of #476.
Do not close #476 when this merges — its Part 1 is still outstanding (below).

Part 1 of #476 is NOT in this PR. "No language model deployment on
memex.systemorph.com has usable quota" is an Azure/deployment provisioning
decision and remains a maintainer action. Nothing here changes model
deployments, portal config, or hard-codes a different model.

What was already fixed, and what was left

#938 and #944 (both merged this morning) covered the requested-vs-serving half:
the round records the model that answered, ThreadMessage.RequestedModelName
carries the one that was asked for, an operator gets MODEL_SUBSTITUTED, and a
round that no model can serve now fails with a localized message rather than
settling Completed. Token attribution follows the same truth via #933/#595.

What none of that reached is the failure the portal actually dies on. It is not a
credential problem: HasUsableCredential proves a key resolves, never that the
deployment will serve. A model with a perfect key still refuses every round when
it is out of quota (429) or when the endpoint is faulting (500) — which is #476's
own repro, the fallback landing on a credentialed model and dying on its rate limit.
Nothing local can predict throttling, so the requirement is not "never fall back onto
a throttled model" — it is fail legibly when the provider refuses.

The defect

ThreadExecution's terminal catch pasted ex.Message into the response cell's Text
and the thread's Summary. For an Azure / System.ClientModel transport failure that
string is the status line, then the JSON body, then the complete HTTP response
header block (x-ms-client-request-id, x-ratelimit-remaining-requests,
Retry-After, …) — in English, whatever the viewer's language. That dump was the
thread's only account of why it failed; and because the round had been silently
substituted, it named a model the user never picked with nothing saying why.

The fix

Same presentation/condition split #944 established, extended to the provider:

ProviderFailureClassifier (new, pure, no provider-SDK dependency) names the
condition off the exception chain — the streaming pipeline wraps provider
faults, so the transport exception is rarely outermost. Typed
HttpRequestException.StatusCode first, else the conventional Status: NNN
banner Azure.Core and System.ClientModel both render. The banner probe is anchored
and bounded to three digits, so "Status: ok" or "Status: 4291 tokens" yields
nothing rather than an invented status.
Prose is built at write time off the round's own AccessContext.Locale —
explicit, never ambient CultureInfo (a round hops schedulers):
chat.modelRateLimited (429) / chat.modelProviderError (5xx), each naming the
model that actually served.
A substituted round adds one sentence (chat.modelSubstitutionNote) naming the
requested model and the one used instead. The swap stays silent on a healthy
round (directive 2026-07-13 — an unusable pin is not actionable mid-thread); on a
failed one it is the difference between an explicable thread and a baffling one.
Unclassified exceptions keep their message verbatim. For a tool fault or a bug
of ours that message is the diagnosis — replacing it would be the real
information loss.

Nothing swallowed, nothing retried: the raw provider text stays on the
LogError(ex, …) that already precedes the terminal write, now joined by a
PROVIDER_REFUSED warning carrying the status, the serving model and the requested
one. No quota probe, no circuit breaker, no watchdog — a refusal is reported, not
worked around.

Also folds the three repeated actualModel ?? effectiveModel ?? request.ModelName
spellings in that block into a single servingModel local, so the cell, the token
satellite and the message cannot drift apart.

Tests

ProviderFailureClassifierTest — 16 cases pinning both directions on
verbatim-shaped Azure dumps: the 429 and the 500 classify, a wrapped chain still
classifies, and an ordinary fault does not (over-classifying erases a real
diagnosis).
ProviderRefusalRoundTest — drives a real round whose substitute has a good
credential and then refuses with the full header dump. Asserts the round ends
Error, the cell text and thread summary carry the localized condition plus the
substitution sentence, no header line reaches either, and the record still
attributes the round to the model that ran.

New file rather than edits to ModelSubstitutionTest, deliberately — #960
(fix/948-round-locale-propagation) is touching that file for locale, and this adds
no assertion it already makes.

Localization keys added to both strings.en.json and strings.de.json
(LocalizationTest gates the pair).

Verification

Branch synced with main (621558c4b). dotnet build -c Release -warnaserror clean
for MeshWeaver.AI, MeshWeaver.Messaging.Hub, MeshWeaver.Documentation,
MeshWeaver.AI.Test.

Suite	Result
`ProviderFailureClassifierTest`	16/16
`ProviderRefusalRoundTest`	1/1
`ModelSubstitutionTest` (regression)	2/2
`LocalizationTest`	33/33
`DocumentationLinkIntegrityTest`	1/1

🤖 Generated with Claude Code

The requested-vs-serving half of #476 shipped this morning (#938, #944): the round records the model that ANSWERED, `ThreadMessage.RequestedModelName` carries the one that was asked for, and a round no model can serve now fails with a localized message instead of settling Completed. What none of that touched is the case the portal actually dies on, because it is not a credential problem at all. `HasUsableCredential` proves a key resolves. It cannot prove the deployment will SERVE — a model with a perfect key still refuses every round when it is out of quota (429) or when the endpoint itself is faulting (500). #476's own repro is exactly that: the fallback picks a credentialed model and dies on its rate limit. So the requirement is not "never fall back onto a throttled model" — nothing local can predict throttling — it is: when the provider refuses, fail LEGIBLY. It did not. `ThreadExecution`'s terminal catch pasted `ex.Message` into the response cell's Text and the thread's Summary. For an Azure / System.ClientModel transport failure that string is the status line, then the JSON body, then the COMPLETE HTTP response header block (`x-ms-client-request-id`, `x-ratelimit-remaining-requests`, `Retry-After`, …) — in English, whatever the viewer's language. The thread's only account of why it failed was an unreadable dump, and because the round had been silently substituted, that dump named a model the user never picked with nothing anywhere saying why. Fix — the same presentation/condition split #944 established, extended to the provider: - `ProviderFailureClassifier` (new, pure, no provider-SDK dependency) names the condition off the exception CHAIN — the streaming pipeline wraps provider faults, so the transport exception is rarely outermost. Typed `HttpRequestException.StatusCode` first; otherwise the conventional `Status: NNN` banner Azure.Core and System.ClientModel both render. The banner probe is anchored and bounded to three digits so prose ("Status: ok") or a longer run ("Status: 4291 tokens") yields nothing rather than an invented status. - `ThreadExecution` builds the prose AT WRITE TIME off the round's own `AccessContext.Locale` — explicit, never ambient `CultureInfo` (a round hops schedulers): `chat.modelRateLimited` (429) / `chat.modelProviderError` (5xx), each naming the model that actually served. - A SUBSTITUTED round adds one sentence, `chat.modelSubstitutionNote`, naming the requested model and the one used instead. The swap stays silent on a HEALTHY round (directive 2026-07-13 — an unusable pin is not actionable mid-thread); on a FAILED one it is the difference between an explicable thread and a baffling one, since the failure names a model the user never chose. - Anything unclassified keeps its own message VERBATIM. For a tool fault or a bug of ours that message IS the diagnosis; replacing it with tidy prose would be the real information loss. Nothing is swallowed and nothing is retried: the raw provider text stays on the `LogError(ex, …)` that already precedes the terminal write, now joined by a `PROVIDER_REFUSED` warning carrying the status, the serving model and the requested one. No quota probe, no circuit breaker, no watchdog — a refusal is reported, not worked around. Also folds the three repeated `actualModel ?? effectiveModel ?? request.ModelName` spellings in that block into one `servingModel` local, so the cell, the token satellite and the message cannot drift apart. Keys added to strings.en.json + strings.de.json (LocalizationTest gates the pair). Tests: `ProviderFailureClassifierTest` pins both directions on verbatim-shaped Azure dumps — the 429 and the 500 classify, a wrapped chain still classifies, and an ordinary fault does NOT (over-classifying would erase a real diagnosis). `ProviderRefusalRoundTest` drives a real round whose substitute has a GOOD credential and then refuses with the full header dump: the round ends Error, the cell and the thread summary carry the localized condition plus the substitution sentence, no header line reaches either, and the record still attributes the round to the model that ran. Part 1 of #476 — a deployment with real quota on memex.systemorph.com — is unaffected by this and remains a maintainer/Azure action. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

This PR addresses issue #476’s “provider refusal” failure mode by classifying provider-side HTTP failures (429 / 5xx) and rendering a localized, user-readable terminal message (instead of pasting raw transport/header dumps into the thread), while also clarifying model substitution on failed rounds.

Changes:

Add ProviderFailureClassifier to detect provider refusal status codes through exception chains and let ThreadExecution render localized, concise failure text (plus substitution note when applicable).
Add integration + unit tests that pin both the classifier boundaries and the end-to-end “provider refuses substituted model” round behavior.
Add new localization keys (EN/DE) and documentation updates (What’s New + provider configuration guidance).

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
test/MeshWeaver.AI.Test/ProviderRefusalRoundTest.cs	New integration test covering a substituted round that fails due to provider refusal and ensuring raw header dumps do not reach thread/cell text.
test/MeshWeaver.AI.Test/ProviderFailureClassifierTest.cs	New unit tests pinning classification behavior for 429/5xx and ensuring non-provider faults are not over-classified.
src/MeshWeaver.Messaging.Hub/Localization/strings.en.json	Adds localized strings for rate-limited/provider-error conditions and substitution note (EN).
src/MeshWeaver.Messaging.Hub/Localization/strings.de.json	Adds localized strings for rate-limited/provider-error conditions and substitution note (DE).
src/MeshWeaver.Documentation/Data/WhatsNew/2026-08-08-legible-provider-refusal.md	New What’s New entry describing the improved failure presentation.
src/MeshWeaver.Documentation/Data/AI/ProviderConfiguration.md	Documents the quota/refusal behavior and how the system now classifies/presents provider failures.
src/MeshWeaver.AI/ThreadExecution.cs	Uses provider-failure classification to avoid leaking raw transport dumps into thread/cell text; logs structured refusal info; unifies serving-model selection via a local.
src/MeshWeaver.AI/ProviderFailureClassifier.cs	New pure classifier that extracts HTTP status from typed exceptions or Azure-style “Status: NNN” banners across the exception chain.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…der-failure

github-actions · 2026-08-08T20:45:52Z