feat(dashboard): stream in-flight response bodies into live logs and interactions

CLAUDE.md

@@ -123,7 +123,7 @@ Full reference: `.env.template` and `config/config.yaml`
 - **Usage tracking:** `USAGE_ENABLED` (true), `ENFORCE_RETURNING_USAGE_DATA` (true), `USAGE_RETENTION_DAYS` (90)
 - **Rate limits:** `RATE_LIMITS_ENABLED` (true; no-op until rules exist). Every rule has a scope: `user_path` (consumer control; subtree with ONE shared counter per rule — per-key limits = give each key its own path), `provider` (caps one configured provider instance across all consumers/models), or `model` (subject `openai/gpt-4o` pins one provider's model, bare `gpt-4o` covers it on any provider; matching case-insensitive). Limits: `max_requests`/`max_tokens` per period (`minute`/`hour`/`day`/custom `period_seconds`, sliding window) plus `concurrent` (period_seconds 0: `max_requests` = max in-flight; realtime sessions hold a slot for the session, batch submissions don't — and batch skips provider/model rules since batch files can mix models). Enforcement covers every model endpoint; user-path breaches return 429 (`code: rate_limit_exceeded`) with `Retry-After`, successes carry `x-ratelimit-{limit,remaining,reset}-{requests,tokens}` from the most-constrained matching rule; cache hits bypass. Saturated providers/models are instead routed around: virtual-model load balancing skips them (like stale inventory), a saturated primary route with configured failover rules skips the primary provider and is served by the sweep (which also skips saturated candidates), and only requests with no viable alternative get 429. Token windows are charged to the provider/model that actually executed (from the usage entry), so accounting stays correct under aliasing/failover. Managed in the dashboard (Rate Limits page: scope selector) / `/admin/rate-limits` (GET/PUT/DELETE + `POST .../reset-one`, `POST .../reset`; requests take `scope`+`subject`, with `user_path` as shorthand for user-path rules), or as infrastructure-as-code under `rate_limits.{user_paths,providers,models}:` in `config.yaml` / `SET_RATE_LIMIT_<PATH>` env vars (`rpm/tpm/rph/tph/rpd/tpd/concurrent=N` compact syntax or a JSON rule array; `__` separates path segments) and `SET_PROVIDER_RATE_LIMIT_<NAME>` (same syntax; suffix underscores become hyphens; model rules are YAML/admin-only). Env replaces the whole YAML entry for the same subject; config-sourced rules are read-only in the dashboard and manual edits win over config seeds, like budgets. Token limits are post-accounted from usage entries, so they require `USAGE_ENABLED=true` (startup warns otherwise) and one request can overshoot a token window. Counters are in-memory per instance (N replicas ≈ N× limit) and reset on restart — budgets remain the durable cross-instance control.
 - **Dashboard live logs:**
-  - `DASHBOARD_LIVE_LOGS_ENABLED` (true): keep enabled for low-latency dashboard previews; set false only when live streams are not needed or memory/socket usage must be minimized.
+  - `DASHBOARD_LIVE_LOGS_ENABLED` (true): keep enabled for low-latency dashboard previews; set false only when live streams are not needed or memory/socket usage must be minimized. With `LOGGING_LOG_BODIES` also enabled, in-flight streamed responses render chunk-by-chunk in the request log and Interactions drawer (throttled `audit.stream` events, published only while a dashboard is connected; partial bodies are never buffered server-side).
   - `DASHBOARD_LIVE_LOGS_BUFFER_SIZE` (10000): effective size is capped at `DASHBOARD_LIVE_LOGS_REPLAY_LIMIT + 1` (older events can never be replayed); lower it below the replay limit only to shrink memory at the cost of more replay resets. Buffered events are compact previews — request/response bodies are never retained in the buffer (connected dashboards get them live; history hydrates from persisted audit entries).
   - `DASHBOARD_LIVE_LOGS_REPLAY_LIMIT` (1000): increase when clients commonly reconnect after long gaps (30+ seconds at high traffic); decrease to reduce replay latency and memory. Also bounds the live log buffer.
   - `DASHBOARD_LIVE_LOGS_HEARTBEAT_SECONDS` (15): decrease to 5-10s when proxies need frequent liveness checks; increase to reduce idle network chatter.

internal/admin/dashboard/static/css/dashboard.css

@@ -3704,6 +3704,23 @@ textarea:focus {
   padding: 8px 0 0;
 }
 
+.audit-pane-pending {
+  display: flex;
+  align-items: center;
+  gap: 8px;
+}
+
+.audit-pane-streaming {
+  display: inline-flex;
+  align-items: center;
+  gap: 7px;
+  padding-left: 4px;
+  font-size: 11px;
+  font-weight: 600;
+  letter-spacing: 0.02em;
+  color: var(--text-muted);
+}
+
 .audit-size-warning {
   margin-top: 8px;
   color: var(--warning);
@@ -3781,6 +3798,15 @@ body.conversation-drawer-open {
   gap: 10px;
 }
 
+.conversation-live-status {
+  display: flex;
+  align-items: center;
+  gap: 10px;
+  padding: 4px 16px 20px;
+  color: var(--text-muted);
+  font-size: 13px;
+}
+
 .chat-message {
   border: 1px solid var(--border);
   border-radius: 10px;

internal/admin/dashboard/static/js/dashboard.js

@@ -198,6 +198,7 @@ function dashboard() {
     conversationMessages: [],
     conversationRequestToken: 0,
     conversationReturnFocusEl: null,
+    conversationLiveEntryId: "",
     bodyPointerStart: null,
 
     _parseRoute(pathname) {

internal/admin/dashboard/static/js/modules/audit-list.js

@@ -321,6 +321,10 @@
                 if (liveState === 'audit.completed' || liveState === 'audit.flushed' || liveState === 'audit.detail') {
                     return false;
                 }
+                // A partial response body means the stream is still running,
+                // regardless of the other completion signals (streamed entries
+                // carry status 200 from the moment headers were committed).
+                if (entry._response_partial) return true;
                 if (entry.status_code !== null && entry.status_code !== undefined && entry.status_code !== '') return false;
                 if (Number(entry.duration_ns || 0) > 0) return false;
                 if (entry.error_type || entry.error_message) return false;
@@ -692,6 +696,8 @@
 
             auditRequestPane(entry) {
                 const data = entry && entry.data ? entry.data : null;
+                const empty = !data || (!data.request_headers && !data.request_body);
+                const pending = empty && this.auditEntryLiveInProgress(entry);
 
                 return {
                     title: 'Request',
@@ -708,8 +714,10 @@
                     body: data && data.request_body,
                     bodyCacheRatioLabel: this.auditCacheRatioPillLabel(entry),
                     promptCacheHighlight: this.auditPromptCacheHighlight(entry),
-                    showEmpty: !data || (!data.request_headers && !data.request_body),
+                    showEmpty: empty && !pending,
                     emptyMessage: 'Request details were not captured.',
+                    showPending: pending,
+                    pendingMessage: 'Waiting for request data…',
                     showTooLarge: !!(data && data.request_body_too_big_to_handle),
                     tooLargeMessage: 'Request body was too large to capture.'
                 };
@@ -718,6 +726,8 @@
             auditResponsePane(entry) {
                 const data = entry && entry.data ? entry.data : null;
                 const errorMessage = this.auditEntryErrorMessage(entry);
+                const empty = !data || (!errorMessage && !data.response_headers && !data.response_body);
+                const pending = empty && this.auditEntryLiveInProgress(entry);
 
                 return {
                     title: 'Response',
@@ -732,8 +742,12 @@
                     headers: data && data.response_headers,
                     showBody: !!(data && data.response_body),
                     body: data && data.response_body,
-                    showEmpty: !data || (!errorMessage && !data.response_headers && !data.response_body),
+                    streaming: !!(entry && entry._response_partial && data && data.response_body) &&
+                        this.auditEntryLiveInProgress(entry),
+                    showEmpty: empty && !pending,
                     emptyMessage: 'Response details were not captured.',
+                    showPending: pending,
+                    pendingMessage: 'Response in progress…',
                     showTooLarge: !!(data && data.response_body_too_big_to_handle),
                     tooLargeMessage: 'Response body was too large to capture.'
                 };

internal/admin/dashboard/static/js/modules/audit-list.test.cjs

@@ -91,6 +91,67 @@ test('auditResponsePane returns the shared response-pane contract', () => {
     assert.equal(pane.tooLargeMessage, 'Response body was too large to capture.');
 });
 
+test('auditEntryLiveInProgress stays true while a partial response body is streaming', () => {
+    const module = createAuditListModule();
+    const streaming = {
+        _live: true,
+        _live_pending: true,
+        _live_state: 'audit.stream',
+        _response_partial: true,
+        status_code: 200,
+        data: { response_body: { choices: [] } }
+    };
+
+    assert.equal(module.auditEntryLiveInProgress(streaming), true);
+    assert.equal(module.auditEntryLiveInProgress({
+        ...streaming,
+        _live_state: 'audit.completed',
+        _response_partial: false,
+        duration_ns: 1000
+    }), false);
+});
+
+test('audit panes surface pending spinners and streaming badges for live rows', () => {
+    const module = createAuditListModule();
+
+    const waiting = { _live: true, _live_pending: true, _live_state: 'audit.updated', data: {} };
+    const waitingResponse = module.auditResponsePane(waiting);
+    assert.equal(waitingResponse.showPending, true);
+    assert.equal(waitingResponse.showEmpty, false);
+    assert.equal(waitingResponse.pendingMessage, 'Response in progress…');
+    const waitingRequest = module.auditRequestPane(waiting);
+    assert.equal(waitingRequest.showPending, true);
+    assert.equal(waitingRequest.showEmpty, false);
+    assert.equal(waitingRequest.pendingMessage, 'Waiting for request data…');
+
+    const streaming = {
+        _live: true,
+        _live_pending: true,
+        _live_state: 'audit.stream',
+        _response_partial: true,
+        status_code: 200,
+        data: { response_body: { choices: [{ index: 0, message: { role: 'assistant', content: 'partial' } }] } }
+    };
+    const streamingPane = module.auditResponsePane(streaming);
+    assert.equal(streamingPane.streaming, true);
+    assert.equal(streamingPane.showBody, true);
+    assert.equal(streamingPane.showPending, false);
+
+    // A stale partial flag on an already-settled entry must not keep the badge.
+    const settled = {
+        ...streaming,
+        _live_state: 'audit.completed',
+        duration_ns: 1000
+    };
+    assert.equal(module.auditResponsePane(settled).streaming, false);
+
+    const persisted = { data: {} };
+    const persistedPane = module.auditResponsePane(persisted);
+    assert.equal(persistedPane.showPending, false);
+    assert.equal(persistedPane.showEmpty, true);
+    assert.equal(persistedPane.streaming, false);
+});
+
 test('audit cache helpers summarize cached prompt usage and derive a preview from the request body', () => {
     const module = createAuditListModule({
         window: {

internal/admin/dashboard/static/js/modules/conversation-drawer.js

@@ -86,19 +86,78 @@
 
                 const requestToken = ++this.conversationRequestToken;
                 this.conversationOpen = true;
-                this.conversationLoading = true;
                 this.conversationError = '';
                 this.conversationAnchorID = entry.id;
                 this.conversationEntries = [];
                 this.conversationMessages = [];
                 document.body.classList.add('conversation-drawer-open');
                 requestAnimationFrame(() => this._focusConversationDrawer());
+
+                // A live entry has no persisted row to fetch yet — render it
+                // from the live preview data and keep re-rendering as stream
+                // events arrive (see refreshLiveConversation).
+                if (this._conversationEntryLivePending(entry)) {
+                    this.conversationLiveEntryId = String(entry.id).trim();
+                    this.conversationLoading = false;
+                    this.applyLiveConversationEntry(entry);
+                    return;
+                }
+                this.conversationLiveEntryId = '';
+                this.conversationLoading = true;
                 await this.fetchConversation(entry.id, requestToken);
             },
 
+            // Guarded like every cross-module call: modules mix optionally.
+            // Without the live-logs module no entry is ever marked _live, so
+            // false is the correct degraded answer.
+            _conversationEntryLivePending(entry) {
+                return typeof this.auditEntryLiveDetailPending === 'function' &&
+                    this.auditEntryLiveDetailPending(entry);
+            },
+
+            applyLiveConversationEntry(entry) {
+                this.conversationEntries = [entry];
+                this.conversationMessages = this.buildConversationMessages([entry], entry.id);
+            },
+
+            // refreshLiveConversation re-renders an open live conversation when
+            // its audit entry merges a new live event. Once the entry is
+            // persisted, the full thread (prior turns, final bodies) is
+            // hydrated from the store instead.
+            refreshLiveConversation(entry) {
+                if (!this.conversationOpen || !this.conversationLiveEntryId || !entry) return;
+                if (String(entry.id || '').trim() !== this.conversationLiveEntryId) return;
+                const state = String(entry._live_state || '').trim();
+                if (state === 'audit.flushed' || state === 'audit.detail') {
+                    this.conversationLiveEntryId = '';
+                    const requestToken = ++this.conversationRequestToken;
+                    this.fetchConversation(entry.id, requestToken);
+                    return;
+                }
+                this.applyLiveConversationEntry(entry);
+            },
+
+            // conversationLiveWaiting reports whether the open live conversation
+            // is still waiting on the in-flight request (drives the drawer's
+            // progress spinner).
+            conversationLiveWaiting() {
+                if (!this.conversationOpen || !this.conversationLiveEntryId) return false;
+                const entry = (this.conversationEntries || [])[0];
+                if (!entry) return true;
+                return typeof this.liveAuditStateSettled !== 'function' ||
+                    !this.liveAuditStateSettled(entry._live_state);
+            },
+
+            conversationLiveStatusText() {
+                return (this.conversationMessages || []).length > 0
+                    ? 'Model is responding…'
+                    : 'Waiting for request data…';
+            },
+
             closeConversation() {
                 this.conversationOpen = false;
                 this.conversationRequestToken++;
+                this.conversationLiveEntryId = '';
                 document.body.classList.remove('conversation-drawer-open');
                 const returnFocusEl = this.conversationReturnFocusEl;
                 this.conversationReturnFocusEl = null;