diff --git a/llm-gateway.mdx b/llm-gateway.mdx index d5f13b5..738a2e9 100644 --- a/llm-gateway.mdx +++ b/llm-gateway.mdx @@ -49,7 +49,7 @@ Every harness has a default tier: one model plus up to five fallbacks. You set i ### Custom tiers -Custom tiers route by request header. You create a tier on the dashboard, give it a header key and value, and pin it to a model with its own fallbacks. When an incoming request carries that header, Manifest sends it to that tier's model instead of the default. +Custom tiers route by request header. You create a tier on the dashboard, give it a header key and value, and pin it to a model with its own fallbacks. When an incoming request carries that header, Manifest sends it to that tier's model instead of the default. The header wins even when the body names an explicit model ID. The header key is yours to choose (lowercase letters, numbers, and hyphens). A few names are reserved and rejected, including `authorization`, `cookie`, and `x-api-key`. Send the header from your client like any other: diff --git a/reference/api.mdx b/reference/api.mdx index 0b25a6e..c0521c5 100644 --- a/reference/api.mdx +++ b/reference/api.mdx @@ -50,7 +50,7 @@ curl -X POST http://localhost:2099/v1/chat/completions \ }' ``` -The body is forwarded verbatim to the resolved provider, with `model` rewritten to the actual model ID. All standard OpenAI fields (`temperature`, `max_tokens`, `tools`, `tool_choice`, `response_format`, `stream`, etc.) pass through. +Manifest adapts the body to the provider it picked. It replaces `model` with the real model ID. It converts the body to the format that provider expects. It renames parameters the provider spells differently, like `max_tokens` and `max_completion_tokens`. And it merges the model parameters saved on your harness into every attempt. Some fields only exist at OpenAI: `stream_options`, `reasoning_effort`, `modalities`, `audio`, and `prediction`. Those reach OpenAI and OpenRouter. Every other provider gets the request without them, and nothing warns you. ## Anthropic messages @@ -77,7 +77,7 @@ curl http://localhost:2099/v1/models \ -H "Authorization: Bearer mnfst_YOUR_KEY_HERE" ``` -Send `auto` to let Manifest route, or send any listed model ID to skip routing and go straight to that provider. If you send a model ID that is not in this harness-specific list, Manifest returns [M302: Model not available](/errors/M302). See [Routing → Route a specific model](/llm-gateway#route-a-specific-model). +Send `auto` to let Manifest route, or send any listed model ID to skip routing and go straight to that provider. If you send a model ID that no connected provider can serve, Manifest returns [M302: Model not available](/errors/M302). See [Routing → Route a specific model](/llm-gateway#route-a-specific-model). ### Inspect model capabilities @@ -185,7 +185,7 @@ Routing and fallback both work with streams. If the primary model fails before t ## Errors -The proxy returns a standard JSON error envelope: +Errors come in two shapes, depending on the caller. A tool or SDK call gets a real HTTP status and the JSON envelope below. A chat or streaming client gets an HTTP `200` that looks like a normal completion, and the error text sits inside the assistant message. ```json { @@ -203,17 +203,17 @@ The proxy returns a standard JSON error envelope: |---|---| | `401` | Invalid or missing `Authorization` header | | `402` | Manifest Free plan quota reached (`error.code = PLAN_LIMIT_REQUESTS`, [M204](/errors/M204)), or provider billing/quota error from the upstream | -| `429` | Manifest rate limit tripped ([M201](/errors/M201), [M202](/errors/M202), [M203](/errors/M203)) | +| `429` | Manifest rate limit tripped ([M201](/errors/M201), [M202](/errors/M202), [M203](/errors/M203)), or a rate limit the provider itself returned | | `5xx` | Upstream provider error (triggers [fallback](/llm-gateway#fallback)) | A [hard limit](/llm-gateway#hard-limits) block ([M200](/errors/M200)) is not in this table: it comes back as an HTTP `200` chat completion whose assistant message carries the block text. When the fallback chain is exhausted, the response keeps the primary model's real error status and carries `X-Manifest-Fallback-Exhausted: true`; the body keeps the provider's own error code, or `fallback_exhausted` when there is none. ## Rate limits -Self-hosted instances default to **100 requests per 60 seconds** per harness. Override with `THROTTLE_TTL` and `THROTTLE_LIMIT` ([Environment variables](/reference/environment-variables)). +The gateway enforces three caps. A workspace can send **200 requests per minute** ([M201](/errors/M201)). An IP can send **500 requests per minute** ([M202](/errors/M202)). A workspace can have **10 requests in flight at once** ([M203](/errors/M203)). The caps are the same on Cloud and self-hosted. No setting changes them. -Cloud rate limits are tied to your plan and shown in the dashboard. +Your Cloud plan sets your monthly request quota ([M204](/errors/M204)). The per-minute caps stay the same on every plan. ## Response headers -Every response carries [routing headers](/reference/headers) so your client can see which model and tier handled the request, without parsing the response body. +Routed responses carry [routing headers](/reference/headers). They tell your client which model and tier handled the request, with no need to parse the response body. A request rejected before routing (bad key, quota, rate limit) carries none. diff --git a/reference/environment-variables.mdx b/reference/environment-variables.mdx index 13b6545..771782f 100644 --- a/reference/environment-variables.mdx +++ b/reference/environment-variables.mdx @@ -36,8 +36,8 @@ Manifest reads its configuration from environment variables. In the bundled Dock | `MANIFEST_ENCRYPTION_KEY` | `BETTER_AUTH_SECRET` | Key used to encrypt stored provider credentials at rest. Falls back to `BETTER_AUTH_SECRET` when unset — set a separate 32+ char value so a session-cookie leak doesn't also decrypt every stored provider key. Set it before first boot | | `MANIFEST_DISABLE_HSTS` | — | Set `1` to silence the boot warning about HSTS being unavailable on a plain-HTTP deployment. Prefer an `https://` `BETTER_AUTH_URL` anywhere reachable from the internet | | `WINGMAN_CORS_ORIGINS` | — | Extra browser origins allowed to call the gateway, comma-separated. Useful when you host your own copy of [Wingman](https://github.com/mnfst/wingman), the open-source gateway tester; the hosted one at `wingman.manifest.build` is always allowed | -| `THROTTLE_TTL` | `60000` | Rate-limit window in milliseconds | -| `THROTTLE_LIMIT` | `100` | Max requests per window per harness | +| `THROTTLE_TTL` | `60000` | Window (ms) for the rate limit on the dashboard's own API calls. Does not apply to gateway traffic on `/v1` | +| `THROTTLE_LIMIT` | `100` | Max dashboard API calls per window. The gateway has its own fixed caps, see [Rate limits](/reference/api#rate-limits) | ## Database diff --git a/reference/glossary.mdx b/reference/glossary.mdx index 6d9377d..a38e222 100644 --- a/reference/glossary.mdx +++ b/reference/glossary.mdx @@ -38,11 +38,11 @@ The model ID you send to opt into routing. When Manifest sees `auto`, it applies ## Custom tier -A routing rule matched on a request header. You choose the header key and value on the dashboard and pin the tier to its own model and fallback chain. When a request carries that header, it routes to the tier's model instead of the default. See [Routing → Custom](/llm-gateway#custom-tiers). +A routing rule matched on a request header. You choose the header key and value on the dashboard and pin the tier to its own model and fallback chain. When a request carries that header, it routes to the tier's model instead of the default. That holds even when the body names an explicit model ID. See [Routing → Custom](/llm-gateway#custom-tiers). ## Direct routing -Sending a real model ID (one that `GET /v1/models` lists) instead of `auto`. Manifest forwards the request straight to that provider and skips tiers and fallbacks. The response carries `X-Manifest-Tier: direct`. +Sending a real model ID (one that `GET /v1/models` lists) instead of `auto`. Manifest forwards the request straight to that provider and skips tiers and fallbacks. The response carries `X-Manifest-Tier: direct`. One exception: if the request also carries a matching custom-tier header, the header wins. ## Fallback @@ -50,7 +50,7 @@ The retry mechanism that kicks in when the primary model fails. Manifest tries t ## Fallback chain -The ordered list of models tried for a single tier, primary first. Up to 5 models. When all of them fail, Manifest returns HTTP `424` with `X-Manifest-Fallback-Exhausted: true`. +The ordered list of models tried for a single tier, primary first. Up to 5 models. When all of them fail, Manifest returns the primary model's real error status, plus the header `X-Manifest-Fallback-Exhausted: true`. ## Provider diff --git a/reference/headers.mdx b/reference/headers.mdx index cfbb737..0753d4f 100644 --- a/reference/headers.mdx +++ b/reference/headers.mdx @@ -14,23 +14,25 @@ Manifest reads a few request headers, and returns a set of `X-Manifest-*` respon | ----------------------- | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | `Authorization` | `Bearer mnfst_` | **Required.** Authenticates the harness. | | `Content-Type` | `application/json` | **Required.** | -| `anthropic-version` | `2023-06-01` | **Required for `/v1/messages`.** Forwarded to the upstream. | +| `anthropic-version` | `2023-06-01` | Accepted and ignored. Anthropic SDKs send it on their own. Manifest picks the version it sends upstream. | | Your custom tier header | your value | Routes the request to a [custom tier](/llm-gateway#custom-tiers). You set the key and value when you create the tier, so the exact header name is up to you. | | `x-session-key` | any string | Groups requests into a session. Manifest keeps a session on the same upstream where the provider supports it, for sticky routing and prompt caching. | -The custom tier header is the only one that changes routing: send the header you configured on a tier and the request goes to that tier's model. See [Routing → Custom](/llm-gateway#custom-tiers). +The custom tier header is the only one that changes routing: send the header you configured on a tier and the request goes to that tier's model. The header wins even when the body names an explicit model ID. See [Routing → Custom](/llm-gateway#custom-tiers). -## Response headers (every request) +## Response headers (routed requests) | Header | Description | Example | | ---------------------------- | -------------------------------------------------------------------------------- | --------------- | -| `X-Manifest-Tier` | The tier that handled the request: `default`, a custom tier's name, or `direct` | `default` | +| `X-Manifest-Tier` | How the request was routed: `standard` after a custom-tier header match, `direct` for an explicit model ID, `simple` when Manifest answered without calling a provider | `direct` | | `X-Manifest-Model` | Model that actually served the response | `claude-sonnet-4-6` | | `X-Manifest-Provider` | Upstream provider | `anthropic` | -| `X-Manifest-Reason` | Why that route was picked: `default`, `header-match`, or `direct` | `header-match` | +| `X-Manifest-Reason` | Why that route was picked, for example `header-match`, `direct`, or `default`. Other values exist | `header-match` | | `X-Manifest-Output-Modality` | Output modality of the response | `text` | | `X-Manifest-Response-Mode` | Whether the response was streamed or `buffered` | `buffered` | +Your custom tier's name appears on the request in the dashboard, never in a header. When Manifest answers a request itself (a spend block, no provider connected, a model it cannot serve), `X-Manifest-Model` and `X-Manifest-Provider` both report `manifest`. + ## Response headers (fallback only) When the primary model fails and Manifest succeeds on a fallback, two extra headers are added: diff --git a/references/decisions.md b/references/decisions.md index 526c10d..48f9805 100644 --- a/references/decisions.md +++ b/references/decisions.md @@ -67,16 +67,16 @@ The issue-verifier reads this to never re-raise a rejected finding. - 2026-08-15 · IMAGE · providers/api-key-providers.mdx · screenshot of Providers → Usage-based under the "Add a provider" steps · status: accepted (FULFILLED — screenshot captured on a live self-hosted instance and shipped as images/providers-usage-based.png in the same PR; supersedes issue #55 findings 5-6 as recorded) → PR pending · https://github.com/mnfst/docs/issues/67 (finding 4) - 2026-08-15 · issue #55 findings 5 and 6 (IMAGE, both still `raised`) are SUPERSEDED by issue #67 finding 4: both ask for shots of screens the platform removed (the Routing-page provider tile grid, RoutingModals.tsx:326; the two-tab API Key / Subscription panel, ProviderSelectContent.tsx:304-305). No reversal derivation owed: neither was ever accepted or rejected, so no recorded position flips. - 2026-08-16 · SCOPED RUN (reference (api, headers, environment variables, glossary) · cron rotation, autonomous, no human) · pages reference/{api,headers,environment-variables,glossary}.mdx · window: full reference surface at platform main 633455b98 + the ~60-day PR set on that surface (#2540 capabilities param, #2566 cost param, #2678, #2565, #2485, #2668, #2569, #2623/#2628 credits-gateway env rename, #2697 pool default, #2539, #2578, and the load-bearing historical #1407 that removed the 424 sentinel on 2026-03-30) · 10 findings filed as issue #68; lastAuditedPr untouched (scoped run), topic stamped 2026-08-16 (first-ever audit of this topic, lastAuditedAt was null). Categories: WRONG ×7, MISSING-CASE ×3. reference/telemetry.mdx deliberately EXCLUDED (owned by the request-logs topic in topics.json). Full loop ran (platform-engineer truth → docs-auditor cross-check → tech-writer drafts → issue-verifier fresh-eyes adversary). Verifier returned PASS-WITH-EDITS and caught TWO factual errors in visible text that all three earlier stages had missed: (a) finding 9's title claimed "failed requests carry no routing headers at all" — false, provider-error and exhausted responses DO carry them (proxy-response-handler.ts:309,404); only pre-routing rejections (auth/quota/gateway-429/internal) carry none, and the wrong version contradicted finding 4 step 5 inside the same issue; (b) finding 3 claimed "the only 429s come from the gateway caps" — false, an upstream provider 429 is forwarded with its real status and full envelope (proxy-response-handler.ts:308-310). Both fixed before filing, plus 7 line-number corrections and the missing "## Findings" skeleton heading. Also killed invented vocabulary in visible text ("the management API" → "the dashboard's own API calls") and tightened finding 8 option 2 back inside scope. Four owned backlog lines consumed (see backlog.md). Coherence GREEN (48 pages, 56 vars). No live reproduction: no harness key exists on the VM, so evidence is code-cited only. -- 2026-08-16 · WRONG · reference/api.mdx:212,214 + reference/environment-variables.mdx:38-39 · the documented gateway rate-limit override does nothing: the proxy controller carries @SkipThrottle() (proxy.controller.ts:119,123) so the throttle env pair (app.module.ts:78-81) governs only Nest-routed dashboard controllers; real gateway caps are hardcoded constants 200/min per tenant (M201), 500/min per IP (M202), 10 in flight (M203) with no env/ConfigService read in proxy-rate-limiter.ts:4-9; the page's "100 req/60s per harness, override with the two vars" is wrong in both directions, and Cloud plans move only the MONTHLY quota (plan-limits.ts:9,11-16). NOT a re-raise of #58 finding 2 (that is errors/M201-M203.mdx saying the caps are unconfigurable; this is the opposite-side defect on the pages that advertise the knob) · status: raised · https://github.com/mnfst/docs/issues/68 (finding 1) -- 2026-08-16 · WRONG · reference/api.mdx:6,204,208 + reference/headers.mdx:47 + reference/glossary.mdx:53 · "424 on fallback exhaustion" on three in-scope pages; no production path emits 424 since PR #1407 (2026-03-30), exhaustion preserves the PRIMARY provider's status (proxy.service.ts:1258,1265) and signals via the exhausted header (proxy-response-handler.ts:403-405); api.mdx:208's "the only status that does not trigger a fallback" describes a rule that does not exist (fallback-status-codes.ts:1-3 is >=400, no exclusions). Fix caveat recorded: the body code is providerCode ?? 'fallback_exhausted' (:410), so it is NOT always fallback_exhausted. Page-echo of #53 finding 2 (llm-gateway.mdx), different pages, named. Consumes backlog line of 2026-08-10 · status: raised · https://github.com/mnfst/docs/issues/68 (finding 2) -- 2026-08-16 · WRONG · reference/api.mdx:191-197,205 · the only error example on the API page shows type "limit_exceeded" + numeric code 429; code is string|null (errorField() nulls non-strings, proxy-error-sanitizer.ts:110) with the numeric status in a SEPARATE field, type comes from a closed OpenAI set, limit_exceeded is a routing reason not a type, and M200 is an HTTP 200 assistant stub (proxy-friendly-response.ts:89,118). Page-echo of #53 finding 3 + #58 finding 1, different page, named. Consumes backlog line of 2026-08-10 · status: raised · https://github.com/mnfst/docs/issues/68 (finding 3) -- 2026-08-16 · WRONG · reference/headers.mdx:27,30 · X-Manifest-Tier is documented as possibly carrying "a custom tier's name"; a header-tier match reports tier 'standard' + reason 'header-match' and the name is DB-only (resolve.service.ts:325-335, buildMetaHeaders has no name branch); Manifest-blocked 200s report tier 'simple' / model 'manifest' / provider 'manifest' (proxy-friendly-response.ts:59-63), values the doc says cannot occur. Fix constraint recorded: do NOT introduce the complexity-routing tier vocabulary, it is being retired in-product (RoutingDeprecationNotice.tsx:4) · status: raised · https://github.com/mnfst/docs/issues/68 (finding 4) -- 2026-08-16 · WRONG · reference/api.mdx:53 · "the body is forwarded verbatim … all standard OpenAI fields pass through" false: 8 OpenAI-only fields are stripped outside the openai/openrouter passthrough set (provider-client-converters.ts:98-107,113,372), max_tokens is renamed both directions, DeepSeek clamps to 8192, cache markers are injected and the payload is converted whole per target (provider-client.ts:572,611,647,724,729), and harness param defaults are merged into every routed attempt (proxy.service.ts:304-317,328-338). The page naming `stream` as passing through makes the sibling stream_options drop maximally surprising · status: raised · https://github.com/mnfst/docs/issues/68 (finding 5) -- 2026-08-16 · WRONG · reference/environment-variables.mdx:32 · the bind-interface row tells Docker users to open the listener for LAN (a no-op: the shipped image already sets it, docker/Dockerfile:78) and to restrict a container by hand-editing the compose ports line (wiped by install.sh --upgrade, which mv's over docker-compose.yml at :239-251 and never touches .env); real reachability is the compose publish line (docker-compose.yml:50) driven by the two commented host-mapping vars (.env.example:28,31). Page-echo of #56 finding 1 (self-hosted.mdx:208), different page, named. RESOLVED stage-1 uncertainty: the container binds 0.0.0.0 · status: raised · https://github.com/mnfst/docs/issues/68 (finding 6) -- 2026-08-16 · WRONG · reference/headers.mdx:17 (+ api.mdx:61 example) · the Anthropic version header is marked "Required for /v1/messages. Forwarded to the upstream" — both halves false: zero inbound readers/validators in packages/backend/src (all 7 hits are outbound injection of the hardcoded 2023-06-01), and the gateway contract test posts to /v1/messages without it and gets 401 auth, not 400 (routes.test.mjs:85-95). Consumes backlog line of 2026-08-10 · status: raised · https://github.com/mnfst/docs/issues/68 (finding 7) -- 2026-08-16 · MISSING-CASE · reference/glossary.mdx:45 + reference/headers.mdx:21 · a matching custom-tier header BEATS an explicit model ID in the body: resolveExplicitModel calls resolveHeaderTier first and returns on match before the catalog lookup (proxy.service.ts:903-905,976-978, rationale :957-959); the common SDK case (mandatory model field + tier header) silently runs on the tier's model and bills it. Rest of the direct-routing definition verified correct (fallback_routes null :1063, scoring bypassed :1057-1072) · status: raised · https://github.com/mnfst/docs/issues/68 (finding 8) -- 2026-08-16 · MISSING-CASE · reference/api.mdx:200-206,218 + reference/headers.mdx:23 · the status table and the "every request" header heading assume one client kind: streaming/chat clients get HTTP 200 with the message in the assistant turn instead of 401/402 (proxy-exception.filter.ts:38-44,131-149), M100/M101/M102/M200/M302 are 200 for EVERYONE and appear in no row, and pre-routing rejections (auth/quota/429/internal) carry no X-Manifest-* headers. Page-echo of #58 finding 3 (errors.mdx), different page, lower severity. Verifier-corrected: provider-error and exhausted responses DO carry the headers (proxy-response-handler.ts:309,404) · status: raised · https://github.com/mnfst/docs/issues/68 (finding 9) -- 2026-08-16 · MISSING-CASE (low severity, honest leave-as-is option 3) · reference/api.mdx:80 · "any model ID not in this harness-specific list returns M302" is a false universal — a provider-qualified/inferable ID still routes through connected credentials before discovery learns it (proxy.service.ts:961-964,990-999), M302 only on the null path (:1438). Third page carrying the concept after llm-gateway.mdx:89 (#53 finding 4) and self-hosted.mdx (#49 finding 3, accepted); named as such. Harm is missed capability, not failure · status: raised · https://github.com/mnfst/docs/issues/68 (finding 10) +- 2026-08-16 · WRONG · reference/api.mdx:212,214 + reference/environment-variables.mdx:38-39 · the documented gateway rate-limit override does nothing: the proxy controller carries @SkipThrottle() (proxy.controller.ts:119,123) so the throttle env pair (app.module.ts:78-81) governs only Nest-routed dashboard controllers; real gateway caps are hardcoded constants 200/min per tenant (M201), 500/min per IP (M202), 10 in flight (M203) with no env/ConfigService read in proxy-rate-limiter.ts:4-9; the page's "100 req/60s per harness, override with the two vars" is wrong in both directions, and Cloud plans move only the MONTHLY quota (plan-limits.ts:9,11-16). NOT a re-raise of #58 finding 2 (that is errors/M201-M203.mdx saying the caps are unconfigurable; this is the opposite-side defect on the pages that advertise the knob) · status: accepted (option 1, batch 2026-08-18; today's truth documented — RATE_MAX_REQUESTS env overrides exist in still-open platform PR #2724, docs follow once released) → PR pending · https://github.com/mnfst/docs/issues/68 (finding 1) +- 2026-08-16 · WRONG · reference/api.mdx:6,204,208 + reference/headers.mdx:47 + reference/glossary.mdx:53 · "424 on fallback exhaustion" on three in-scope pages; no production path emits 424 since PR #1407 (2026-03-30), exhaustion preserves the PRIMARY provider's status (proxy.service.ts:1258,1265) and signals via the exhausted header (proxy-response-handler.ts:403-405); api.mdx:208's "the only status that does not trigger a fallback" describes a rule that does not exist (fallback-status-codes.ts:1-3 is >=400, no exclusions). Fix caveat recorded: the body code is providerCode ?? 'fallback_exhausted' (:410), so it is NOT always fallback_exhausted. Page-echo of #53 finding 2 (llm-gateway.mdx), different pages, named. Consumes backlog line of 2026-08-10 · status: accepted (batch 2026-08-18; api.mdx + headers.mdx halves were already fixed by merged PR #62, glossary.mdx:53 fixed now) → PR pending · https://github.com/mnfst/docs/issues/68 (finding 2) +- 2026-08-16 · WRONG · reference/api.mdx:191-197,205 · the only error example on the API page shows type "limit_exceeded" + numeric code 429; code is string|null (errorField() nulls non-strings, proxy-error-sanitizer.ts:110) with the numeric status in a SEPARATE field, type comes from a closed OpenAI set, limit_exceeded is a routing reason not a type, and M200 is an HTTP 200 assistant stub (proxy-friendly-response.ts:89,118). Page-echo of #53 finding 3 + #58 finding 1, different page, named. Consumes backlog line of 2026-08-10 · status: accepted (batch 2026-08-18; mostly pre-fixed by merged PR #62, the provider-forwarded-429 clause added now) → PR pending · https://github.com/mnfst/docs/issues/68 (finding 3) +- 2026-08-16 · WRONG · reference/headers.mdx:27,30 · X-Manifest-Tier is documented as possibly carrying "a custom tier's name"; a header-tier match reports tier 'standard' + reason 'header-match' and the name is DB-only (resolve.service.ts:325-335, buildMetaHeaders has no name branch); Manifest-blocked 200s report tier 'simple' / model 'manifest' / provider 'manifest' (proxy-friendly-response.ts:59-63), values the doc says cannot occur. Fix constraint recorded: do NOT introduce the complexity-routing tier vocabulary, it is being retired in-product (RoutingDeprecationNotice.tsx:4) · status: accepted (option 1, batch 2026-08-18; LIVE-VERIFIED: a Manifest-answered request returns X-Manifest-Tier: simple, model/provider manifest, reason no_provider/model_not_available) → PR pending · https://github.com/mnfst/docs/issues/68 (finding 4) +- 2026-08-16 · WRONG · reference/api.mdx:53 · "the body is forwarded verbatim … all standard OpenAI fields pass through" false: 8 OpenAI-only fields are stripped outside the openai/openrouter passthrough set (provider-client-converters.ts:98-107,113,372), max_tokens is renamed both directions, DeepSeek clamps to 8192, cache markers are injected and the payload is converted whole per target (provider-client.ts:572,611,647,724,729), and harness param defaults are merged into every routed attempt (proxy.service.ts:304-317,328-338). The page naming `stream` as passing through makes the sibling stream_options drop maximally surprising · status: accepted (option 1 compact, batch 2026-08-18) → PR pending · https://github.com/mnfst/docs/issues/68 (finding 5) +- 2026-08-16 · WRONG · reference/environment-variables.mdx:32 · the bind-interface row tells Docker users to open the listener for LAN (a no-op: the shipped image already sets it, docker/Dockerfile:78) and to restrict a container by hand-editing the compose ports line (wiped by install.sh --upgrade, which mv's over docker-compose.yml at :239-251 and never touches .env); real reachability is the compose publish line (docker-compose.yml:50) driven by the two commented host-mapping vars (.env.example:28,31). Page-echo of #56 finding 1 (self-hosted.mdx:208), different page, named. RESOLVED stage-1 uncertainty: the container binds 0.0.0.0 · status: accepted (already fixed by merged PR #65 — BIND_ADDRESS row re-pointed and HOST_BIND_ADDRESS row added; nothing left to do) → PR pending · https://github.com/mnfst/docs/issues/68 (finding 6) +- 2026-08-16 · WRONG · reference/headers.mdx:17 (+ api.mdx:61 example) · the Anthropic version header is marked "Required for /v1/messages. Forwarded to the upstream" — both halves false: zero inbound readers/validators in packages/backend/src (all 7 hits are outbound injection of the hardcoded 2023-06-01), and the gateway contract test posts to /v1/messages without it and gets 401 auth, not 400 (routes.test.mjs:85-95). Consumes backlog line of 2026-08-10 · status: accepted (option 1, batch 2026-08-18) → PR pending · https://github.com/mnfst/docs/issues/68 (finding 7) +- 2026-08-16 · MISSING-CASE · reference/glossary.mdx:45 + reference/headers.mdx:21 · a matching custom-tier header BEATS an explicit model ID in the body: resolveExplicitModel calls resolveHeaderTier first and returns on match before the catalog lookup (proxy.service.ts:903-905,976-978, rationale :957-959); the common SDK case (mandatory model field + tier header) silently runs on the tier's model and bills it. Rest of the direct-routing definition verified correct (fallback_routes null :1063, scoring bypassed :1057-1072) · status: accepted (option 1, batch 2026-08-18; the parked llm-gateway.mdx:52 sentence fixed in the same PR with the same wording) → PR pending · https://github.com/mnfst/docs/issues/68 (finding 8) +- 2026-08-16 · MISSING-CASE · reference/api.mdx:200-206,218 + reference/headers.mdx:23 · the status table and the "every request" header heading assume one client kind: streaming/chat clients get HTTP 200 with the message in the assistant turn instead of 401/402 (proxy-exception.filter.ts:38-44,131-149), M100/M101/M102/M200/M302 are 200 for EVERYONE and appear in no row, and pre-routing rejections (auth/quota/429/internal) carry no X-Manifest-* headers. Page-echo of #58 finding 3 (errors.mdx), different page, lower severity. Verifier-corrected: provider-error and exhausted responses DO carry the headers (proxy-response-handler.ts:309,404) · status: accepted (option 1, batch 2026-08-18; headers heading retitled to routed requests) → PR pending · https://github.com/mnfst/docs/issues/68 (finding 9) +- 2026-08-16 · MISSING-CASE (low severity, honest leave-as-is option 3) · reference/api.mdx:80 · "any model ID not in this harness-specific list returns M302" is a false universal — a provider-qualified/inferable ID still routes through connected credentials before discovery learns it (proxy.service.ts:961-964,990-999), M302 only on the null path (:1438). Third page carrying the concept after llm-gateway.mdx:89 (#53 finding 4) and self-hosted.mdx (#49 finding 3, accepted); named as such. Harm is missed capability, not failure · status: accepted (option 1, batch 2026-08-18) → PR pending · https://github.com/mnfst/docs/issues/68 (finding 10) - 2026-08-16 · cleared near-findings from the reference audit (recorded so they are not re-derived): the model-list capabilities/cost query params are CORRECT on every falsifiable point (proxy.controller.ts:104-116,150-168,190,194); X-Manifest-Confidence/-Specificity set but unlisted is acceptable curation; the pool-default and gateway-tester-leak rows stay #48 findings 8/10 (re-verified unchanged, NOT re-raised); the stream-idle-timeout var is absent from the page but also absent from the Docker env example, so nothing is used-but-undefined; the autofix-timeout default 10000 is right; the recordings-retention default 365 is right FOR THIS PAGE'S self-hosted audience even though the code default is plan-derived; PORT/NODE_ENV/BETTER_AUTH_URL/recordings-path documented as the bundled compose's EFFECTIVE values, upholding the 2026-08-12 verdict; api.mdx:26 pointing at the Harnesses page for a key is imprecise (key lives at the harness's Settings tab, Settings.tsx:247-311) but lands nobody on a dead screen; the glossary describing only Default+Custom tiers is correct under "less beats confusing" since complexity routing is being retired; the session-key row is true as written (the momentum/cache cliff would need a concept no page defines); the content-type "Required" row is true in practice (body parsing is content-type driven, main.ts:206-207); api.mdx:184 streaming/fallback is correct; glossary "Up to 5 models" is true (routing.dto.ts:201-202,212-213). INTERNAL-LEAK sweep of the four pages for every codename in the stage-1 internal-tools table: 0 hits beyond the already-open #48 finding 10 rows. - 2026-08-15 · cleared near-finding (no issue): platform PR #2700 made a stale fallback entry deletable (a tier/fallback row pointing at a de-listed model or disconnected provider used to fail with "Cannot resolve fallback model ..." on every save). llm-gateway.mdx:110-124 never claimed otherwise, so the platform moved TOWARD the docs. Recorded so a future run does not re-derive it. - 2026-08-17 · WRONG · reference/telemetry.mdx:14 · "Inline image data is stripped out" reads as covering stored logs; the image redaction only feeds the throwaway copy used to pick a model (proxy.controller.ts:382, proxy.service.ts:249,266) and a stored body is kept verbatim, uncapped, unscrubbed (attempt-recording-capture.ts:19-54; 2 MiB spec) · status: raised · https://github.com/mnfst/docs/issues/69 (finding 1) @@ -117,3 +117,4 @@ The issue-verifier reads this to never re-raise a rejected finding. - 2026-08-18 · SCOPE DECISION (upheld by the verifier) · a seventh candidate was found and deliberately NOT raised: nine deploy pages tell users to move to a custom domain and frame it as an OAuth concern only, never saying the public-URL var must follow the domain or every login on it fails, with the setup screen unusable as a way back in once an admin exists. Real, but nine pages sharing one variable name is the deploy topic's own audit, not a window whose single PR touched an operator README and a CI workflow. Parked in backlog.md with the nine line numbers, the four verbatim-vs-reworded split, and the koyeb exception (its public URL is templated to the platform domain, so it has no such note and is not an omission) · owner: deploy platforms topic - 2026-08-18 · MISSING-CASE · reference/api.mdx:33,37 · the endpoints table's Responses row plus the format-translation sentence under it are silent on the one thing that breaks a Responses client: translation carries messages, tools and tool results, not Responses-only conversation state. A `/v1/responses` turn is served natively only when the resolved endpoint's format is chatgpt (provider-client.ts:273); every other target is rebuilt as a chat body whose allow-list (responses-adapter.ts:115-133) has no previous_response_id, the route validates nothing inbound (proxy.controller.ts:213-219), and Manifest reads the field nowhere (one repo-wide hit, responses-adapter.ts:391, an output hardcoded to null). The reply still looks continuable (status completed, fresh synthetic id at :379/:638, store false at :393), so a threading conversation silently restarts from zero with no error, header or log line. PREDATES the window's PR: #2732 only made the surrounding class quieter, turning a provider 400/422 into a silent drop; the lossy translation shipped with the endpoint itself (first release 5.55.1). Distinct from the still-open #68 finding 5 on the same page (that one is field stripping on /v1/chat/completions at :53), so both should ship in one PR · status: raised · https://github.com/mnfst/docs/issues/79 (finding 1) - 2026-08-18 · cleared near-findings from the PR #2732 run (recorded so they are not re-derived): the window's PR is a bugfix with NO doc claim to correct, and that verdict stands. A self-hosted version note ("upgrade past the release that fixes this") is NOT shippable: the fix is unreleased, no tag contains the merge commit, the next number is inference, and after the cut it would be pure bug-history narration, which AUDIENCE principle 1 forbids by name. llm-gateway.mdx:87 ("a direct model ID works on all three endpoints") is CLEAN: direct routing is identical on the three endpoints and the translation loss is not a routing property. The Responses row's "Use it for" column is CORRECT, verified against the platform's own responses-only model pattern one family at a time, Codex exception included. The "you must use a Responses client for those models" reading of that row was CLEARED, not raised: the platform transparently swaps those models onto the provider's Responses endpoint even when the client posted to /v1/chat/completions, the column header is a recommendation not a requirement, and llm-gateway.mdx:87 already answers it. Reasoning items were deliberately excluded from the finding: dropping them is the only correct mapping to a chat target, no doc sentence claims otherwise, and the word has zero hits in the whole corpus. CASE-MATRIX sweep: deployment, install state, agents and activation all collapse (translation is stateless, no env var, no setting, no migration, no per-agent read); the only dimension that does not collapse is providers, which is exactly what the finding raises. INTERNAL-LEAK sweep of the two in-scope pages and of the filed issue body against the stage-1 internal list (the healing service codename, its four env vars including the healing endpoint's internal subdomain, and the automated PR-reviewer bot named in the PR description): 0 hits. Noted for future runs: stage 1 listed Autofix itself as internal, which is WRONG, AUDIENCE.md names it as product vocabulary and /autofix is a published page +- 2026-08-18 · parked items consumed with the #68 PR: llm-gateway.mdx:52 gains the header-beats-model-ID sentence (same wording as #68 finding 8), self-hosted.mdx optional-env list drops 'rate limits' (same root as #68 finding 1). diff --git a/self-hosted.mdx b/self-hosted.mdx index 17a0223..2573d12 100644 --- a/self-hosted.mdx +++ b/self-hosted.mdx @@ -325,7 +325,7 @@ The first two are required and the installer generates both, so a default instal The installer also generates `MANIFEST_ENCRYPTION_KEY`. If you are installing by hand, set it too. Left unset, Manifest falls back to `BETTER_AUTH_SECRET` for at-rest encryption and warns on every boot — meaning one leaked session-signing secret also decrypts every stored provider key and OAuth token. Set it **before first boot**: introducing it later means re-encrypting what is already in the database. -Everything else is optional: rate limits, provider timeouts, email delivery for alerts and password resets, OAuth logins, connection-pool sizing, recording storage and retention, Autofix, and Sentry. See [Environment variables](/reference/environment-variables) for the full list with defaults. +Everything else is optional: provider timeouts, email delivery for alerts and password resets, OAuth logins, connection-pool sizing, recording storage and retention, Autofix, and Sentry. See [Environment variables](/reference/environment-variables) for the full list with defaults. `NODE_ENV` and `SEED_DATA` are fixed by the bundled compose file and are not knobs for a self-hosted install. The image is a production artifact, and the demo-data seeder refuses to run under `NODE_ENV=production` whatever `SEED_DATA` says — use the first-run setup wizard to create your admin account.