feat(session): add session keeping with sticky load balancing and threaded audit logs

.env.template

@@ -27,6 +27,18 @@
 # TAGGING_HEADER_1_DELIMITER=,
 # TAGGING_HEADER_2=X-Internal-Routing
 # TAGGING_HEADER_2_DONOTPASS=true
+
+# Session keeping: identify which requests belong to one client session, for sticky
+# virtual-model load balancing and audit-log session grouping. On by default with a
+# built-in registry of session headers/body fields known coding tools send (Claude
+# Code, Codex CLI, OpenCode, Kilo Code, Goose, …) plus content-based auto-detection
+# for untagged chat requests. Extra headers are numbered from 1 and merged over the
+# built-ins; the optional TRANSFORM "session-uuid" extracts a session_<uuid> value.
+# SESSION_KEEPING_ENABLED=true
+# SESSION_AUTO_DETECT=true
+# SESSION_BUILTIN_RULES=true
+# SESSION_HEADER_1=X-My-Session
+# SESSION_HEADER_1_TRANSFORM=
 # Log output format: leave unset to auto-detect, or set to "json" / "text"
 # LOG_FORMAT=text
 # Log verbosity: "debug", "info" (default), "warn", or "error"

CLAUDE.md

@@ -117,8 +117,9 @@ Full reference: `.env.template` and `config/config.yaml`
   - `REALTIME_ENABLED` (true: Expose the realtime speech-to-speech websocket at `/v1/realtime` and the `/p/{provider}/v1/realtime` upgrade. The canonical `/v1/realtime` route needs only `REALTIME_ENABLED`; the `/p/{provider}/v1/realtime` upgrade additionally requires passthrough routes enabled (`ENABLE_PASSTHROUGH_ROUTES`) with the provider listed in `ENABLED_PASSTHROUGH_PROVIDERS`. The gateway is a transparent websocket reverse proxy — it injects provider credentials and relays the provider's realtime event schema verbatim (no translation), so clients connect without provider API keys. Only providers implementing realtime accept sessions. Currently: OpenAI and xAI/Grok Voice Agent (both `wss://…/v1/realtime`); Z.ai/Zhipu GLM-Realtime (`wss://…/api/paas/v4/realtime`); Bailian/Qwen-Omni (`wss://dashscope…/api-ws/v1/realtime`); and Azure OpenAI (`wss://<resource>/openai/realtime?api-version=…&deployment=…`, `api-key` header). All use OpenAI's realtime event schema (Z.ai adds extensions that relay transparently). Provider-specific notes: xAI voice models (e.g. `grok-voice-latest`) aren't in upstream `/models` discovery, so configure them via `XAI_MODELS`, and xAI bills realtime per-minute (no token usage reported); Azure realtime requires a realtime-capable `AZURE_API_VERSION` (the default may be too old) and the model selects the Azure deployment. (MiniMax was evaluated but skipped — its conversational realtime schema is not OpenAI-compatible.) Sessions are gated by the same model-access and budget rules as other model endpoints; usage is tracked per `response.done` event, accepting both the OpenAI singular and Alibaba plural token-detail spellings. The same flag also exposes the OpenAI-compatible WebRTC surface (via the optional `core.RealtimeCallProvider` interface — OpenAI and xAI at the shared `…/v1/realtime/{calls,client_secrets}` shape, and Azure OpenAI at its GA `<resource>/openai/v1/realtime/{calls,client_secrets}` surface with `api-key` auth and no api-version; xAI gates WebRTC calls per team, so unauthorized accounts get the upstream 403 relayed while client_secrets works. Bailian is deliberately not wired: its WebRTC is allowlist-only with a per-customer endpoint provided by sales, plus no call id in the answer; Z.ai has no WebRTC realtime): `POST /v1/realtime/calls` exchanges SDP (raw `application/sdp` offer with `?model=`, or multipart `sdp` + `session` JSON fields; the session/query model is rewritten to the resolved provider model so aliases and virtual models work) and relays the answer with a gateway-relative `Location: /v1/realtime/calls/{call_id}` header; `POST /v1/realtime/client_secrets` mints ephemeral browser credentials routed by `session.model` (falling back to the nested transcription model); and `GET /v1/realtime?call_id=…` attaches to an existing call as a sideband websocket (an in-memory per-instance call registry recalls the route for calls created through the same instance — 6h TTL, capped; otherwise pass explicit `model`+`provider` params). WebRTC media and events flow directly between client and provider, so after creating a call the gateway attaches its own best-effort sideband observer websocket to record usage per `response.done` (entries carry endpoint `/v1/realtime/calls`; skipped when usage tracking is off, and gateway-relayed sideband attaches for registry-known calls don't tap usage to avoid double counting). WebRTC signaling counts toward request-scoped rate limits, but concurrent-scope rules can't span a WebRTC call's lifetime since only signaling transits the gateway; ephemeral client secrets authenticate clients directly against the provider, so those sessions bypass the gateway entirely and are untracked.)
 - **Storage:** `STORAGE_TYPE` (sqlite), `SQLITE_PATH` (default: `data/gomodel.db` when a `./data` directory exists — existing deployments, Docker; otherwise the OS per-user data dir, e.g. `~/.local/share/gomodel/gomodel.db` — see `internal/platformdir`; the local model cache resolves `.cache` vs the OS cache dir the same way), `POSTGRES_URL`, `MONGODB_URL`. `/v1/responses` snapshots and `/v1/conversations` history persist to the configured backend (30-day TTL, hourly sweep); the in-memory fallback stores are byte-capped and used only by embedded setups that skip app wiring.
 - **Models:** `MODELS_ENABLED_BY_DEFAULT` (true), `KEEP_ONLY_ALIASES_AT_MODELS_ENDPOINT` (false), `CONFIGURED_PROVIDER_MODELS_MODE` (`fallback` or `allowlist`, default `fallback`; `allowlist` skips upstream `/models` for providers with configured lists); persisted overrides restrict/allow selectors with `user_paths`. When alias-only models listing is enabled, `GET /v1/models` returns only model aliases, not full concrete model specs, to operators.
-- **Virtual models:** Redirects (aliases / load balancers) and access policies are managed in the admin dashboard and persisted to the `virtual_models` store. A redirect with one target is a plain alias; a redirect with several targets is load balanced by `strategy`: `round_robin` (default; rotates across targets, honoring per-target `weight`) or `cost` (always routes to the cheapest catalog-priced available target, falling back to the first target when none are priced). Unavailable targets are skipped, so a redirect works while any target is live. Virtual models can also be declared as infrastructure-as-code under `virtual_models:` in `config.yaml` or via the `VIRTUAL_MODELS` env var (a JSON array; env merges over YAML, winning per `source`). Declarative entries are validated at startup, override admin-store rows with the same `source`, and are read-only in the dashboard. Startup validation is catalog-independent: structure plus explicit target `provider` names — a name matching no configured provider (a typo) aborts startup listing the registered providers; a name declared under `providers:` but unregistered (e.g. credentials unset in this environment) only warns and the target stays unavailable; target *model* availability is never a startup gate (checked at resolve time, since the catalog loads asynchronously).
+- **Virtual models:** Redirects (aliases / load balancers) and access policies are managed in the admin dashboard and persisted to the `virtual_models` store. A redirect with one target is a plain alias; a redirect with several targets is load balanced by `strategy`: `round_robin` (default; rotates across targets, honoring per-target `weight`) or `cost` (always routes to the cheapest catalog-priced available target, falling back to the first target when none are priced). Unavailable targets are skipped, so a redirect works while any target is live. Requests carrying a detected session id (see Session keeping) stick to the target that served the session first (`session_affinity`, default true; explicit `false` restores stateless balancing — flag on YAML/env/admin API and the editor's "Session keeping" checkbox); a pinned target that turns unavailable/saturated is re-picked by the strategy and the session re-pins, the all-saturated first-target 429 fallback never pins, and pins are per-instance with a 6h idle TTL. Virtual models can also be declared as infrastructure-as-code under `virtual_models:` in `config.yaml` or via the `VIRTUAL_MODELS` env var (a JSON array; env merges over YAML, winning per `source`). Declarative entries are validated at startup, override admin-store rows with the same `source`, and are read-only in the dashboard. Startup validation is catalog-independent: structure plus explicit target `provider` names — a name matching no configured provider (a typo) aborts startup listing the registered providers; a name declared under `providers:` but unregistered (e.g. credentials unset in this environment) only warns and the target stays unavailable; target *model* availability is never a startup gate (checked at resolve time, since the catalog loads asynchronously).
 - **MCP gateway:** `MCP_ENABLED` (true: expose the MCP-protocol endpoints; a no-op until servers are declared). GoModel aggregates upstream MCP (Model Context Protocol) servers behind the authenticated streamable-HTTP endpoint `/mcp` (POST JSON-RPC, GET notification stream, DELETE session end) and per-server endpoints `/mcp/{server}`. On `/mcp`, tools and prompts are namespaced `{server}_{name}` with deterministic ordering; `tools/call` accepts the namespaced name (longest server-prefix match) or a unique bare name; `/mcp/{server}` exposes original names. Tools, prompts, resources, and resource templates relay with raw schemas/results verbatim; upstream `instructions` are merged into the gateway's `initialize` result. Servers come from three sources with the usual precedence: `mcp.servers:` map in `config.yaml`, the `MCP_SERVERS` env var (JSON object merged over YAML per name), and the `mcp_servers` admin store (dashboard MCP Servers page / `/admin/mcp-servers` GET/PUT/DELETE + `POST .../{name}/reconnect` + `GET .../{name}/catalog` for the per-server tools/prompts/resources inspector); declarative entries are validated at startup, shadow same-name store rows, and are read-only in the dashboard (secret header values are redacted as `***` in admin reads, and a `***` value on PUT preserves the stored secret). Per-server fields: `url` + `transport` (`http` streamable default, `sse` legacy), or declarative-only `stdio` (`command`/`args`/`env` — rejected via admin API/dashboard because runtime-registered subprocesses would be an RCE vector), `headers` (upstream credentials, `${ENV}` supported; the gateway is a credential boundary — client bearer tokens are never forwarded upstream), `allowed_tools`/`disallowed_tools`, `user_paths` (visibility subtree scoping like virtual models — filtered out of `tools/list`, not just blocked at call time), `tool_timeout` (30s default). The `X-MCP-Servers` request header narrows a session to a comma-separated server subset. One upstream session is shared per server (lazy dial, redial-once on death); a failed listing marks the server `degraded` keeping its last catalog (stale carry-forward, 60s re-probe, 5m re-list, `list_changed` notifications trigger resync). Downstream sessions are SDK-managed (`Mcp-Session-Id`, 30m idle timeout), bound to the initializing user path (a different principal presenting the session ID gets 404), and each session sees a visibility-filtered tool snapshot taken at initialize. Every MCP POST is gated by user-path rate limits and budgets; every `tools/call` writes a usage entry (`provider="mcp"`, `provider_name`=server, `model`=namespaced tool, duration/sizes/error in raw data, labels/user_path as usual) and MCP paths are audit-logged model interactions whose entries are labelled with the JSON-RPC method (tool/prompt name for calls) and `provider="mcp"`, so request-log and live-log rows are self-describing; with `LOGGING_LOG_BODIES` the JSON-RPC request and response frames (SSE replies decoded) are captured on POST entries too. Server→client MCP features (sampling, elicitation, roots) and resource subscriptions are not negotiated in v1. Spec: `docs/dev/2026-07-07_mcp-gateway-spec.md`.
+- **Session keeping:** `SESSION_KEEPING_ENABLED` (true), `SESSION_AUTO_DETECT` (true), `SESSION_BUILTIN_RULES` (true), numbered `SESSION_HEADER_<N>` (+`_TRANSFORM`, only `session-uuid`) env vars / `session:` block in `config.yaml` (headers merged over builtins per name; credential headers rejected). Every model-interaction request gets a session id via `internal/session`: explicit headers win (built-in registry: `X-Session-Id`, `X-Claude-Code-Session-Id`, `Session-Id`/`Session_id`, `X-Litellm-Session-Id`, `Helicone-Session-Id`, `Agent-Session-Id`), then body signals (`metadata.user_id` with the session-uuid transform for both Claude Code formats, `session_id`, `litellm_session_id`, `prompt_cache_key`, `conversation`/`conversation.id`), then content auto-detection for chat/responses (sha256 of model + system/instructions + tools + leading messages through the first user turn + user path → `auto-<hex16>`; stable as turns append). Non-UUID ids are scoped by user path. The id rides the request context (`core.SessionIDFromContext`), drives virtual-model session affinity, and is persisted as the indexed `session_id` column on audit entries (also on live `auditPreview` events). The Audit Logs dashboard groups entries into session threads by default ("Group by session" toggle, localStorage-persisted): `GET /admin/audit/sessions` lists threads (latest entry + count + span; filters apply to entries before grouping; sessionless entries are singleton threads keyed by their own id), `GET /admin/audit/log?session_id=…` (works without date params) fetches one thread. `/v1/responses` `previous_response_id` chaining is not a session signal yet (the `conversation` field is).
 - **Tagging:** Every request can be labelled from configured HTTP headers. Rules are managed in the dashboard (Settings → "Tagging based on headers", persisted to the `tagging_settings` store) or declared as infrastructure-as-code under `tagging.headers:` in `config.yaml` / numbered env vars `TAGGING_HEADER_1=X-My-Tags` with optional `TAGGING_HEADER_1_PREFIX` (trimmed from each extracted label only), `TAGGING_HEADER_1_DONOTPASS` (default false: headers are forwarded as-is; true strips the header before provider forwarding on passthrough/realtime routes — translated routes never forward client headers), and `TAGGING_HEADER_1_DELIMITER` (default `,`; one header value can carry several labels). An env entry replaces the whole YAML entry with the same header name (unset companion vars reset fields to defaults rather than inheriting YAML values); declarative entries override admin-store rows and are read-only in the dashboard. Credential-bearing headers (`Authorization`, `Cookie`, API-key headers, …) are rejected as tagging sources. Managed API keys can also carry labels (`labels` on `POST /admin/auth-keys`, replaceable later via `PUT /admin/auth-keys/{id}/labels` where `[]` clears, or API Keys → Create API Key / Edit Labels in the dashboard); every request authenticated with the key gets them, merged and de-duplicated with header-extracted labels. Labels are recorded on usage entries (`labels`) and audit log entries (`data.labels`). The dashboard usage page shows a by-label breakdown (`GET /admin/usage/labels`) and label chips with a label filter on the request log (`label` query param on `GET /admin/usage/log`).
 - **Audit logging:** `LOGGING_ENABLED` (true), `LOGGING_LOG_BODIES` (true), `LOGGING_LOG_AUDIO_BODIES` (false: refines `LOGGING_LOG_BODIES` for audio endpoints — base64 audio for both `/v1/audio/speech` output and `/v1/audio/transcriptions` upload (≤8 MB each, else `too_large`) + dashboard playback, plus transcription upload metadata; no effect unless `LOGGING_LOG_BODIES` is on, in which case audio-off records a placeholder), `LOGGING_LOG_HEADERS` (true), `LOGGING_RETENTION_DAYS` (30)
 - **Usage tracking:** `USAGE_ENABLED` (true), `ENFORCE_RETURNING_USAGE_DATA` (true), `USAGE_RETENTION_DAYS` (90). Callers can read their own status without admin access via `GET /v1/usage`: usage summary over a date window (`start_date`/`end_date`/`days`, default last 30 days UTC) plus budget and rate-limit statuses, all scoped to the caller's effective user path (managed key binding, else the user-path header).