You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
src/review/screenshot-table-gate.ts (:91-143) only checks the MARKDOWN STRUCTURE of a PR's screenshot table (does a table exist, does each row contain ![]()/<img> markup) — it never looks at the actual image content. A contributor can satisfy this gate by pasting two identical images, an unrelated screenshot, or a screenshot with no visible before/after difference at all, and the gate has no way to catch it. This is a real gap in a check the team already treats as load-bearing for frontend PRs (see the "frontend PR screenshots non-negotiable" policy).
The self-host GPU now runs a vision-language model (qwen3-vl:8b-instruct, wired via env.AI_VISION, #4335) that's idle capacity outside of confirmed-visual-regression calls. The existing visual pipeline (src/review/visual/capture.ts, src/review/visual/pixel-diff.ts) already fetches and stores the before/after images this gate would need — no new capture infrastructure required.
Requirements
Add an ADVISORY-only vision check (mirror visual-findings.ts's existing pattern: pure decision logic separate from the live caller, never a gate blocker on its own) that verifies a PR's screenshot-table images are: (a) not pixel-identical to each other, (b) plausibly relevant to the stated change (not an obviously unrelated image).
Must degrade gracefully exactly like the existing visual-vision advisory: no AI_VISION configured, a fetch failure, or an unparseable response all mean "no finding," never a thrown error or a false gate block.
Config-gated (global + per-repo, matching every other gate/settings field's precedence) — off by default until validated on real PRs.
Deliverables
New advisory check wired into the screenshot-table gate path, config-gated
Unit tests for the pure decision logic (mirroring visual-findings.test.ts's coverage bar) and the live caller wiring (mirroring visual-vision-wiring.test.ts)
Manual validation against a handful of real past PRs (both genuine and gamed screenshot tables) with results recorded in the PR description
Expected outcome
The screenshot-table gate can catch identical/irrelevant screenshot submissions it currently cannot see at all, using GPU capacity that's already idle between confirmed-regression vision calls.
Part of #4325.
Context
src/review/screenshot-table-gate.ts(:91-143) only checks the MARKDOWN STRUCTURE of a PR's screenshot table (does a table exist, does each row contain![]()/<img>markup) — it never looks at the actual image content. A contributor can satisfy this gate by pasting two identical images, an unrelated screenshot, or a screenshot with no visible before/after difference at all, and the gate has no way to catch it. This is a real gap in a check the team already treats as load-bearing for frontend PRs (see the "frontend PR screenshots non-negotiable" policy).The self-host GPU now runs a vision-language model (
qwen3-vl:8b-instruct, wired viaenv.AI_VISION, #4335) that's idle capacity outside of confirmed-visual-regression calls. The existing visual pipeline (src/review/visual/capture.ts,src/review/visual/pixel-diff.ts) already fetches and stores the before/after images this gate would need — no new capture infrastructure required.Requirements
visual-findings.ts's existing pattern: pure decision logic separate from the live caller, never a gate blocker on its own) that verifies a PR's screenshot-table images are: (a) not pixel-identical to each other, (b) plausibly relevant to the stated change (not an obviously unrelated image).env.AI_VISION(the binding Stretch: evaluate spare GPU headroom for a local vision-language model in visual-review #4335 already built) — do not build a second vision binding.AI_VISIONconfigured, a fetch failure, or an unparseable response all mean "no finding," never a thrown error or a false gate block.Deliverables
visual-findings.test.ts's coverage bar) and the live caller wiring (mirroringvisual-vision-wiring.test.ts)Expected outcome
The screenshot-table gate can catch identical/irrelevant screenshot submissions it currently cannot see at all, using GPU capacity that's already idle between confirmed-regression vision calls.
Effort
M