fix(ci): judge Activity decode overhead by an interval, not 5 pairs

.github/workflows/ci.yml

@@ -841,6 +841,7 @@ jobs:
           python3 scripts/webui/verify_generated_key_helper_tests.py
           python3 scripts/webui/verify_reverse_proxy_tests.py
           python3 scripts/webui/verify_single_model_tests.py
+          python3 scripts/webui/activity_performance_verdict_tests.py
           python3 scripts/webui/verify_activity_performance_tests.py
           python3 scripts/webui/summarize_activity_evidence_tests.py
           python3 scripts/webui/activity_gate_host_tests.py

Makefile

@@ -1000,6 +1000,7 @@ verify-webui-helper-tests: ## Run pure-Python WebUI verifier helper tests; no se
 	@$(WEBUI_BUNDLE_PY) scripts/webui/verify_generated_key_helper_tests.py
 	@$(WEBUI_BUNDLE_PY) scripts/webui/measure_startup_tests.py
 	@$(WEBUI_BUNDLE_PY) scripts/webui/verify_single_model_tests.py
+	@$(WEBUI_BUNDLE_PY) scripts/webui/activity_performance_verdict_tests.py
 	@$(WEBUI_BUNDLE_PY) scripts/webui/verify_activity_performance_tests.py
 	@$(WEBUI_BUNDLE_PY) scripts/webui/summarize_activity_evidence_tests.py
 	@$(WEBUI_BUNDLE_PY) scripts/webui/activity_gate_host_tests.py

docs/webui-integration-matrix.md

@@ -14,7 +14,7 @@ This matrix is the canonical handoff for the bundled WebUI release gate. Mock/un
 | State completion and races | Existing contract/unit suites plus `verify-webui-rust` cover bad transitions, operation list, restart/gap fixtures, download cancellation and schema errors; the real-router harness additionally exercises empty-library rescan, real same-port fresh server restart SSE, event-ring gap SSE, stale revision 409, unknown-field 400 and unsupported terminal-operation cancel on the production secured route. | PASS for the real-router fake harness at `ff4463c6`; unit/aggregate rerun remains root-owned. | Preserve failed attempts and exact commit/binary identity in the final evidence table. |
 | Browser visual/accessibility engines | `pnpm --dir webui run browser` keeps the approved Chromium pixel-baseline/Chromium-AX suite, while `pnpm --dir webui run browser:all` uses `playwright.engines.config.ts` for Chromium/Firefox/WebKit DOM, keyboard, layout and axe coverage without invoking Chromium-only CDP or screenshot baselines. CI installs all three engines and runs both paths through `verify-webui-frontend`. | Root PASS on mock/Vite Chromium evidence: 28 default Chromium tests, 7 cross-surface/performance Chromium tests in `integration-1848/engines-f873-quiet-pass`, `chromium-prefix-default-pass`, and latest true DOM+paint timer evidence in `integration-1848/engines-browser-only-timing-pass`; 311 Vitest, 17 Node, 49 contract tests, typecheck and lint also passed. | **Native Safari on the installed artifact: USER-REPORTED PASS 2026-09-16**, against `mlxcel-server --webui` serving the embedded bundle over a loopback-forwarded session, not a Vite preview. Scope confirmed by the maintainer, and only this scope: screen and layout reviewed by eye across Models, Chat, Activity and Settings; keyboard Tab and Shift-Tab order with visible focus and Escape closing dialogs and sheets with focus restored to the opener; Safari native page zoom at 200% with no horizontal page scroll at 390 px, and readability retained with increased contrast. **Manual VoiceOver: user-deferred 2026-09-16, NOT RUN.** It was not part of the Safari pass and no other gate substitutes for it, since the axe and DOM checks in CI assert markup rather than what a screen reader announces. Deferred by the maintainer on the stated grounds that the UI has more pressing changes outstanding, so this does not block #1848 or the epic. Scope of what remains unverified: whether VoiceOver announces the primary navigation and selected route, the command dialog title, the model combobox, status badges, streaming summaries and error text. Lifting the deferral needs a VoiceOver session against the installed artifact; no automated gate can close this row. All-three-engine CI is covered by the `WebUI bundle` job. This supersedes the 2026-09-13 report only for the surfaces above; that one covered the design-system candidate under a Vite preview at `#gallery`. |
 | Bundle budgets and deterministic assets | `make verify-webui-bundle` still enforces initial JS, total JS, embedded bundle size, deterministic checked-in assets, no source maps/local paths/active external origins, and license notice. | PASS on the integrated source (main `b6d69bcc` plus the #1900 docs/CLI branch, no asset change): CI `WebUI bundle` job (https://github.com/lablup/mlxcel/actions/runs/38785163375/job/116595293970) ran `build_bundle.py --verify` and printed `WebUI bundle verified: digest=b4087a9339fa572f260745cf5cd2af5ed9622b51353f6cab1532099072d332be`. The committed manifest records `source_digest_sha256` `1ebf0921089e00db6bf288cdd66fb2ba6265f9ee43a4224a07d483f1201d79c8`, 855,324 embedded bytes (limit 5,242,880), initial JS gzip 181,473 bytes (limit 204,800) and total JS gzip 182,086 bytes (limit 716,800). The CUDA `test-fast` `mlxcel-server` (SHA-256 `575f67ed7a87e7afb3bcdd431247d64d2e93e2295b2777816700a7d8675efcaa`) and `mlxcel` (`3ca9cdd875d5bb676346c8b6627bb531deb0bf2c4db014d4b4e15a9fe5cd6a17`) built for #1900 embed these hashed assets. | A release rebuild must rerun `make verify-webui-bundle` and cite its own job if any WebUI source or asset changes after this row. The initial JS gzip limit has since moved from 204,800 to 262,144 bytes (256 KiB) for the ui-common 0.2 foundations; the figures in this row predate that change. |
-| Performance and large data | Existing frontend tests cover 1,000-entry catalog and 10,000-token transcript parsing; `make verify-webui-startup` records installed empty-router startup/RSS; `make verify-webui-activity-performance` runs the real Activity overhead/native-hidden harness with a checkpoint, including the genuine Linux GB10 headed Chrome minimize-to-`document.hidden` path when invoked with the approved host/display setup. | Root PASS evidence: installed startup/RSS `integration-1848/startup-52103-pass`; mock/Vite Chromium true DOM+paint timing `integration-1848/engines-browser-only-timing-pass` recorded cold 59.8 ms, navigation 25.8 ms, 1,000-entry search 25.9 ms, 10,000-token new conversation 32.0 ms, CLS 0. Prior visible Activity evidence is partial, tied to its source commit only. The GB10 job now runs the headed visible-mode observation overhead on every change (`--perf-mode visible-only-headed`) and summarizes it with `--expect-hidden-native not-run`, so the summary declares `status: incomplete` and `hidden_native: not-run` rather than reading as a complete acceptance. | **Native hidden acceptance: user-deferred 2026-09-16.** Scope of the deferral is the minimize-to-`document.hidden` measurement only; the headed visible-mode overhead is measured in CI and every numeric budget still applies to it. Reason, established on the GB10 runner and not inferred: Xvfb and openbox give headed Chromium a real 700x900 content area, the browser reports the window state as `minimized`, and `document.visibilityState` stays `visible` for 15 seconds anyway; foregrounding a sibling tab in the same context also leaves `document.hidden` false; both visibility-suppressing launch arguments (`--disable-backgrounding-occluded-windows`, `--disable-renderer-backgrounding`) were removed and their removal verified by reading a live browser process's arguments. The unverified step in that chain is whether Playwright's sibling page is a tab in the same window or a separate window, which would make the tab datum uninformative. This is a property of the automation browser, not of the runner or the product. The visibility backoff the row exists to protect stays covered by two unit tests in `webui/src/state/sync.test.ts` and the `visibilitychange` case in `webui/tests/browser.spec.ts`. Lifting the deferral means running `--perf-mode full` again, which needs a browser that propagates a real window-visibility change. |
+| Performance and large data | Existing frontend tests cover 1,000-entry catalog and 10,000-token transcript parsing; `make verify-webui-startup` records installed empty-router startup/RSS; `make verify-webui-activity-performance` runs the real Activity overhead/native-hidden harness with a checkpoint, including the genuine Linux GB10 headed Chrome minimize-to-`document.hidden` path when invoked with the approved host/display setup. Each Activity mode's verdict (#1925) judges the paired decode degradation `(off - on) / off` by its median and a distribution-free 98% sign-test interval: `investigate` (a slowdown) when the interval lies wholly above the 2% budget, `within-target` when it does not and its upper bound is at most 4%, otherwise more alternating pairs, from 12 in steps of 2 up to 40, and then `inconclusive` (measurement noise, not a slowdown). The constants live in `webui/scripts/activity-performance-policy.json`, shared by the harness and both Python validators, which re-derive or cross-check the verdict against `webui/scripts/activity-performance-fixtures.json`; baseline CV and the paired range are diagnostics only. | Root PASS evidence: installed startup/RSS `integration-1848/startup-52103-pass`; mock/Vite Chromium true DOM+paint timing `integration-1848/engines-browser-only-timing-pass` recorded cold 59.8 ms, navigation 25.8 ms, 1,000-entry search 25.9 ms, 10,000-token new conversation 32.0 ms, CLS 0. Prior visible Activity evidence is partial, tied to its source commit only. The GB10 job now runs the headed visible-mode observation overhead on every change (`--perf-mode visible-only-headed`) and summarizes it with `--expect-hidden-native not-run`, so the summary declares `status: incomplete` and `hidden_native: not-run` rather than reading as a complete acceptance. | **Native hidden acceptance: user-deferred 2026-09-16.** Scope of the deferral is the minimize-to-`document.hidden` measurement only; the headed visible-mode overhead is measured in CI and the decode-overhead verdict still applies to it. Reason, established on the GB10 runner and not inferred: Xvfb and openbox give headed Chromium a real 700x900 content area, the browser reports the window state as `minimized`, and `document.visibilityState` stays `visible` for 15 seconds anyway; foregrounding a sibling tab in the same context also leaves `document.hidden` false; both visibility-suppressing launch arguments (`--disable-backgrounding-occluded-windows`, `--disable-renderer-backgrounding`) were removed and their removal verified by reading a live browser process's arguments. The unverified step in that chain is whether Playwright's sibling page is a tab in the same window or a separate window, which would make the tab datum uninformative. This is a property of the automation browser, not of the runner or the product. The visibility backoff the row exists to protect stays covered by two unit tests in `webui/src/state/sync.test.ts` and the `visibilitychange` case in `webui/tests/browser.spec.ts`. Lifting the deferral means running `--perf-mode full` again, which needs a browser that propagates a real window-visibility change. |
 | Actual inference/media | No committed mock gate can satisfy actual inference. Accepted Chat source9385 dense/hybrid/VLM evidence may be referenced only for the specific dense/hybrid/VLM generation paths whose source stayed identical through baseline6976 (`git diff --name-only 9385bcb7 6976e1df -- src Cargo.toml Cargo.lock build.rs webui/src webui/package.json webui/pnpm-lock.yaml` was empty). This is source-equivalence evidence for those paths, not an identical-binary or blanket current-branch runtime/UI claim: #1848 also includes production URL-prefix/public-health changes (`d6b7`, `c629`) plus test harness changes. | Root single-model installed four-arm Hello capture PASS in `integration-1848/single-model-52103-pass`, with semantic review scope limited to that small real checkpoint prompt and legacy own SIGINT `-2` noted; source-equivalent Chat dense/hybrid/VLM rows remain accepted only with the limitation above. | Root must still run or reference exact rows for small public download, VLM image, and genuine native-hidden Activity overhead with `MLXCEL_REQUIRE_MODELS=1`, recording checkpoint revisions, outputs, lifecycle order, and resource before/after. |
 | CUDA host smoke | `make verify-webui-cuda WEBUI_CUDA_EVIDENCE=...` validates the root-run CUDA evidence JSON for UI-on, UI-off and installed-artifact rows. | `not-run`; not satisfied by macOS. | GB10/CUDA host must run UI-on/UI-off server smoke and record result; historic outage waivers are not blanket permission. |
 | Settings / Chat / Activity cross-surface integration | Existing merged child gates provide source-specific evidence; this matrix points to final rerun rather than inheriting a fictional one-run pass. | Prior root notes: Models source22f19 lifecycle/profile/navigation/removal passed with generationTested=false; Chat source9385 dense/hybrid/VLM semantics, Stop active→0, CSP, unload passed; Activity visible paired overhead partial passed while native hidden did not run. | Final integrated artifact must rerun or explicitly reference accepted prior evidence by commit/source and limitation. Chat composer caps new turns at 100 while history import/storage permits 200 per conversation; Stop refreshes aggregate Activity, not a per-request cancellation receipt. |