Skip to content

feat(review): advisory-only AI-vision analysis of before/after visual captures #4111

Description

@JSONbored

Part of #3607 (visual-capture convergence epic). Builds on the gate-wiring issue (screenshotTableGate bot-capture input) but is independently shippable — vision findings surface as a normal advisory finding regardless of whether that gate integration has landed yet.

Context

Both AI-review message-construction paths are currently string-only, text content blocks only: src/services/ai-review.ts::callAiProvider and src/selfhost/ai.ts::toMessages. src/selfhost/ai.ts is the actually-LIVE production path — ai-review.ts's own comments describe its Workers-AI-oriented path as "fully retired — see CONVERGENCE_RUNBOOK.md." Prioritize selfhost/ai.ts.

ai-review.ts has (as of 2026-07-07 — re-verify via grep before editing, line numbers drift fast) at least 4 distinct message-construction call sites needing a text|image content-block union, not just the main reviewer call: the main call (~935-936), two more (~1166, ~1181-1182), and CRITICALLY the tie-break judge (~1600-1601, runDualAiTieBreakJudgeCall/DualAiTieBreakVerdict) that resolves a SPLIT verdict between two AI reviewers. Missing the tie-break judge means a split visual verdict silently falls back to text-only reasoning — grep for every content: message-construction site before considering this done, don't assume the count or line numbers above are still accurate.

Requirements

  • Add a text|image content-block union to both message-construction paths, prioritizing selfhost/ai.ts.
  • Gate image attachment behind the EXISTING pixel-diff threshold already computed by src/review/visual/pixel-diff.ts, so an unchanged route never costs vision tokens.
  • Route the cost through existing cost-control mechanisms rather than inventing a new unbounded spend: check src/review/submitter-reputation.ts (should a low-reputation/burst submitter skip vision, same as other AI neurons?) and BYOK (aiReview.byok/provider/model in .gittensory.yml — vision calls should plausibly ride the maintainer's own BYOK key, since Workers AI is retired).
  • Ship as ADVISORY-ONLY: a new visualFindings field surfaced in the unified PR comment, feeding into the SAME dual-AI-consensus gate any other finding uses. Do NOT make this a new independent blocking mechanism, and do NOT remove or weaken any existing manual-review requirement as part of this issue — that's an explicit non-goal here (any such change is its own separate, much later, data-gated decision, not part of this issue's scope).
  • Consider whether a visual regression should also feed the existing deterministic slop-score detector (src/signals/slop.ts) — a broken layout/unstyled component/placeholder content is exactly the kind of low-effort-submission signal that detector targets. Optional, note as a stretch if time-constrained.

Deliverables

  • Content-block union added to selfhost/ai.ts::toMessages (priority) and ai-review.ts::callAiProvider (all 4+ call sites, tie-break judge included)
  • Pixel-diff-threshold gating so unchanged routes cost zero vision tokens
  • Cost routed through submitter-reputation + BYOK
  • visualFindings advisory field feeding the existing dual-AI-consensus gate
  • Tests: image attached above threshold, omitted below/absent; BYOK routing; reputation-gated skip; the tie-break judge specifically receiving images when the two reviewers split

Expected outcome

Visual regressions get real AI analysis, surfaced as an ordinary advisory finding — fixing the actual current pain (the gate is presence-only and disconnected from real capture) independent of vision-quality tuning, without touching the manual-review hold.

Effort

L — the largest piece of the epic; multiple call sites across 2 files, cost-control integration, and careful advisory-only scoping.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.visualUI/web visual work — owner-led, NOT for Gittensor contributors (extensions excepted)

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions