feat(gemma4): tunable per-request vision image-token budget

[glennneuber]

Summary

Expose the Gemma 4 vision image-token budget as per-request options — image_min_tokens / image_max_tokens on api.Options (in the Runner group) — defaulting to 40 / 1120 via DefaultOptions. This lets it be tuned per request (e.g. from Open WebUI's params) instead of being fixed.

visionServerArgs reads the values and passes them to llama-server as --image-min-tokens / --image-max-tokens, which override llama.cpp's gemma4v defaults (set_limit_image_tokens(40, 280)) via custom_image_min/max_tokens (clip-model.h). Min is clamped to max because llama-server refuses to load when image_max_pixels < image_min_pixels.

Restores the Open WebUI workflow of the old (removed) model.go feature, but Go-only on current main — no clip.cpp patch.

Semantics (important)

These are Runner (load-time) options, so they behave exactly like num_ctx: set them per request, and if the value differs from the loaded runner, the scheduler reloads the runner with the new flags. There is no per-image budget in the mtmd /completion contract, so a true no-reload per-request budget would require patching llama.cpp — deliberately out of scope.

Blast radius

• api/types.go — 2 additive Runner fields + DefaultOptions defaults (parsed by the existing generic reflect.Int path).
• llm/llama_server.go — visionServerArgs reads opts + clamps; 1 call site.
• server/sched.go — unchanged; reload-on-change is handled by the existing reflect.DeepEqual over Runner.

Empirical validation

Built against b9888 and tested with gemma4:e2b-it-q4_K_M (image tokens = prompt_eval_count − text baseline). Per-request options honored, with reload-on-change confirmed in the log (sched.go:263 msg=reloading runner; distinct launches 40/1120, 40/560, 300/1120):

Request	image tokens
2048² image, default	~1091
2048² image, image_max_tokens: 560	~530 (max honored)
64² image, image_min_tokens: 300	~333 (min floor honored, vs ~51 default)

Generation is not regressed

Real descriptions at the raised budget (~1091 tokens, 4× the 280 reference) are coherent and accurate, stopping naturally (done=stop) — no garbage/crash (the position-embedding-OOB risk did not materialize). Text-only unaffected.

Testing

go test ./api/ ./llm/   # incl. TestVisionServerArgs: defaults, custom budget, min>max clamp

Commits

1. Raise the ceiling to 1120 (hardcoded launch flag).
2. Add explicit --image-min-tokens.
3. Make both a per-request api.Options knob (this makes 1–2 the default case).

🤖 Generated with Claude Code