Skip to content

feat: add bounded local streaming chat - #1894

Merged
inureyes merged 9 commits into
mainfrom
feature/issue-1845-streaming-chat
Sep 15, 2026
Merged

inureyes merged 9 commits into
mainfrom
feature/issue-1845-streaming-chat

Conversation

@inureyes

@inureyes inureyes commented Sep 14, 2026 •

Copy link
Copy Markdown
Member

Changes

  • Add local streaming Chat through the existing authenticated inference transport, with frozen turn identity/parameters, canonical Settings defaults and validated single-next-turn overrides, IME-safe composition, explicit partial cancellation, safe Markdown and non-executing reasoning/tool inspection.
  • Reuse published ui-common adapters. Keep history memory-only unless explicitly enabled, bound IndexedDB/imports and local images, and enforce the actual inference media/body contract without persisting credentials.

Verified acceptance

  • Measured source: 9385bcb7ef64386180ea62b3324540eef713a4e2, integrated on merged Settings, Activity and Models. Reports-only follow-up does not change the implementation.
  • Passed: 266 Vitest tests, 17 Node helper tests, type/lint, 49 contracts, 18 constructed browser response shapes, deterministic bundle, both test-fast executables and 28 local browser tests. Workspace all-target Clippy passed with WebUI enabled/disabled at 988493fc; Rust and served asset bytes remain identical at the measured source, except manifest metadata and test/harness changes.
  • Real isolated Llama dense, Granite hybrid and Qwen3-VL image chat passed on M1 Ultra/macOS 27.0. Greeting replies and the image's three colored shapes were semantically reviewed. All three observed active requests above zero, UI Stop, cancelled state and active zero; explicit unload observed worker exit, and the server exited 0. Actual CSP, completed/cancelled axe/layout and no-external-request checks passed.
  • Source CI run 34922431893 passed every executed job, including OpenXLA compile; three conditional skips remain skips. The initial real attempt's Playwright screenshot empty-style false positive and nondurable evidence were corrected and rerun, not counted as acceptance or attributed to GPU/axe/product failure.

Remaining epic scope

Broad final workspace Metal verification belongs to #1848. GPU timeout root-cause investigation and Safari/VoiceOver/native 200% manual checks remain explicitly deferred, not passed. Bilingual reports include exact binary/checkpoint hashes and evidence boundaries.

Closes #1845. Refs #1834.

@inureyes inureyes added status:review Under review type:enhancement New features, capabilities, or significant additions priority:high High priority area:inference Generation, sampling, decoding (incl. speculative, DRY) labels Sep 14, 2026
@inureyes
inureyes force-pushed the feature/issue-1845-streaming-chat branch 2 times, most recently from 132be64 to ac93afd Compare September 15, 2026 01:36
Add the authenticated Chat workflow with frozen turn identities, partial cancellation, separate reasoning and inert tool inspection, bounded local image admission, safe lazy Markdown, and explicit IndexedDB history consent. Project actual resolved image and inference body limits through bootstrap and prevent aborted readers from resolving as successful EOF.

Frontend unit/type/lint and targeted CPU media/whole-bootstrap tests pass. Shared Settings defaults and observation-only selection wiring are coordinated with #1846 and #1847 before final acceptance; hosted browser and real dense/hybrid/VLM checks remain pending.

Refs #1845, #1834.
Preserve text-only chat when image admission is disabled, reject falsely completed imported turns, and validate current image limits across the whole replayed transcript. Fence late stop observations, use the actual shared theme tokens, and add regression evidence plus bilingual review reports.

The independent review fixes pass targeted frontend tests. Settings defaults, observation-only selection integration, hosted browser evidence and isolated real-model acceptance remain explicit pre-merge gates owned by the epic integration sequence.

Refs #1845, #1834.
Node's ESM loader requires explicit JSON import attributes in the Playwright fixture module. Add those attributes and provide a canonical model-scoped runtime response so model selection tests do not manufacture an unrelated observation failure.

Playwright test discovery runs without launching a browser. The original hosted module-load failure is retained in the validation log; actual browser execution remains a hosted gate.

Refs #1845.
Hosted browser checks reached both Chat workflows and found an undersized Settings link plus shrinking privacy checkboxes at 390 pixels. Give local navigation links an explicit target box and prevent checkbox flex shrink while keeping the shared semantic controls and screenshot thresholds unchanged.

The original failure recorded 20 passing browser checks and two failing Chat target-size checks. Hosted browser validation will rerun on the corrected bundle; no local browser or GPU execution is claimed.

Refs #1845.
Add explicit component fixtures for context rejection, authentication expiration, unavailable servers and empty completed transports. Each retains an error outcome without automatic retry or a false completed response. Refresh only the bundle source manifest; production asset bytes are unchanged.

All 144 frontend unit tests, TypeScript and ESLint pass locally. Production-equivalent source 9e9f0cd passed 22 hosted browser checks and deterministic bundle verification. Shared Settings/Activity integration and real-model acceptance remain pending.

Refs #1845.
Use canonical Settings validation and memory-only defaults while freezing each dispatched turn. Consume overrides only after local request admission, preserve rejected drafts, and extend real acceptance with served CSP, accessibility and observed Stop lease checks.
Embed the final Chat, Models, Settings and Activity union after their canonical schema and state integrations. Preserve deterministic assets and strict production acceptance gates.
@inureyes
inureyes force-pushed the feature/issue-1845-streaming-chat branch from ac93afd to 988493f Compare September 15, 2026 02:35
Ignore only trim-empty style attributes left by Playwright screenshot caret cleanup, retaining strict nonempty inline-style rejection. Persist private synthetic reply, screenshot and Stop evidence before later assertions so the list reporter does not discard it.
Record the measured source, real dense/hybrid/VLM semantic and Stop verification, exact checkpoint provenance and corrected screenshot instrumentation. Preserve deferred full Metal and native accessibility gates.
@inureyes inureyes added status:done Completed and removed status:review Under review labels Sep 15, 2026
@inureyes
inureyes merged commit 6976e1d into main Sep 15, 2026
16 checks passed
@inureyes
inureyes deleted the feature/issue-1845-streaming-chat branch September 15, 2026 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:inference Generation, sampling, decoding (incl. speculative, DRY) priority:high High priority status:done Completed type:enhancement New features, capabilities, or significant additions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(webui): implement streaming chat with explicit local privacy

1 participant