Skip to content

feat(selfhost): local-inference binding for advisory-tier AI capabilities (AI_ADVISORY) #4364

Description

@JSONbored

Part of #4325.

Context

Only visual-vision (#4335) got a dedicated local-inference binding (AI_VISION, src/queue/processors.ts:8193-8210) — every other AI-assisted capability shares ONE provider chain (env.AI, built by resolveConfiguredProviderNames/routeProviders, src/selfhost/ai.ts:1021-1178). That chain is deliberately frontier-only (claude-code/codex subscriptions) because it also backs the main review, which drives real gate blockers.

Several capabilities on that SAME shared chain are advisory-only (never gate, never block) and are explicitly documented as not needing a frontier model:

  • AI slop advisory (src/services/ai-slop.ts:18-19, hard-guaranteed non-blocker, :8-10)
  • E2E test-gen (src/services/ai-e2e-test-gen.ts, malformed output silently dropped)
  • Issue planner (src/review/planner.ts, rare/on-demand)
  • AI summaries/rewrite (src/services/ai-summaries.ts, always falls back to deterministic text)

Now that the self-host GPU host is live (RTX A5000), routing these through a local Ollama model instead of the claude-code/codex subscription chain saves real recurring cost without touching the main review's accuracy at all.

Requirements

  • A new dedicated binding, env.AI_ADVISORY, wired in src/server.ts mirroring AI_EMBED/AI_VISION's exact pattern (AI_ADVISORY_BASE_URL/AI_ADVISORY_MODEL/AI_ADVISORY_API_KEY env vars, createOpenAiCompatibleAi). src/server.ts is excluded from Codecov, matching the existing two bindings' treatment.
  • Config-as-code, not a blunt on/off: whether a given capability actually USES AI_ADVISORY (vs. staying on the shared frontier chain) must be config-driven — a global default in the root .gittensory.yml, overridable per-repo in {owner}__{repo}/.gittensory.yml, following the SAME deep-merge precedence every other gate/settings field already uses (see src/signals/focus-manifest.ts). Do not hardcode which capabilities use it — this must be toggleable per capability, per repo, without a code change.
  • Fail-safe identical to AI_VISION/AI_EMBED: absent binding ⇒ falls back to the existing shared env.AI chain, byte-identical to today for any deployment that doesn't set AI_ADVISORY_BASE_URL.
  • This issue is the BINDING + config plumbing only. Routing each of the four candidate capabilities through it is out of scope here — track as its own focused follow-up per capability (start with slop advisory, the highest-volume/highest-savings candidate).

Deliverables

  • env.AI_ADVISORY binding wired in src/server.ts, mirroring AI_EMBED/AI_VISION
  • Global + per-repo config toggle (.gittensory.yml schema field, deep-merge precedence, documented in config/examples/gittensory.full.yml)
  • .env.example documents the three new env vars
  • npm run selfhost:env-reference regenerated (server.ts is a scanned source root)
  • Unit tests for the config resolution (global default + per-repo override, both directions)

Expected outcome

A reusable, config-driven local-inference path that any advisory-only AI capability can opt into per-repo, with zero risk to the main review's accuracy since it's additive and falls back cleanly when unconfigured.

Effort

M

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

Status
Done

Relationships

None yet

Development

No branches or pull requests

Issue actions