Part of #4325.
Non-blocking — explicitly does not gate any other sub-issue or the cutover itself.
Context
The GPU sized for the RAG embedding workload has substantial headroom beyond what the current embedding model needs. The visual-review pipeline's AI-vision analysis step is a plausible candidate to move from a cloud call to a local vision-language model running on this same spare capacity — but this is a genuine quality/cost tradeoff that needs real evaluation, not an assumed win.
Requirements
- Identify 1-2 candidate open vision-language models that fit comfortably in the available VRAM alongside the embedding model running concurrently.
- Run the same set of PR screenshots through both the current cloud-based analysis and the candidate local model, and compare output quality side by side — this issue should produce a quality comparison, not just a "it runs" confirmation.
- Explicitly weigh latency and quality against the cloud path's cost — a local model that's meaningfully worse should be rejected here, not shipped because the hardware happened to be available.
Deliverables
Expected outcome
A data-backed answer to whether the spare GPU capacity is worth using for visual-review analysis — and if not, that's a complete, successful outcome for this issue, not a failure.
Effort
M
Part of #4325.
Non-blocking — explicitly does not gate any other sub-issue or the cutover itself.
Context
The GPU sized for the RAG embedding workload has substantial headroom beyond what the current embedding model needs. The visual-review pipeline's AI-vision analysis step is a plausible candidate to move from a cloud call to a local vision-language model running on this same spare capacity — but this is a genuine quality/cost tradeoff that needs real evaluation, not an assumed win.
Requirements
Deliverables
Expected outcome
A data-backed answer to whether the spare GPU capacity is worth using for visual-review analysis — and if not, that's a complete, successful outcome for this issue, not a failure.
Effort
M