Skip to content

Disclose that agentDryRun does not stop AI/LLM spend #5286

Description

@JSONbored

Context

agentDryRun's doc comment ("records what it WOULD do without performing any GitHub mutation") and the live maintainer-dashboard hint ("Record what the agent would do, without performing it") both imply a safe, cost-free simulation. In reality every AI/LLM spend gate checks mode === "paused" exclusively — dry_run passes straight through and every AI-provider call executes for real, at real cost. Only the terminal GitHub write is suppressed. This is a deliberate, in-code-documented design choice (tagged #token-bleed-spend-gate in ai-review-orchestration.ts:262-265, agent-orchestrator.ts:236, processors.ts:6593,6877) — the defect is that the field name, JSDoc, and product UI copy never disclose it.

A maintainer who leaves this toggle on believing it's cost-free racks up unbounded, recurring real AI-provider spend with zero product-surface indication it's happening. This is the same shape as a real prior incident already in project memory: gate.enabled: false was believed to be a full spend kill-switch but only ever gated a downstream write, not the spend itself.

Requirements

  • Do not change the spend-gate logic — the dry-run-still-spends design is deliberate (token-bleed-spend-gate tag) and out of scope here.
  • Fix the copy at all three surfaces a maintainer actually reads, in priority order.

Deliverables

  • apps/gittensory-ui/src/components/site/app-panels/maintainer-settings.tsx:563-564 — change the toggle hint to something like "Suppress GitHub writes only — AI review calls still run and still cost tokens." Highest priority — this is the copy a maintainer sees when flipping this on to "test safely."
  • src/types.ts:1091-1093 — reword the agentDryRun doc comment to state explicitly this is NOT a cost-free preview; AI/LLM calls still execute and still incur provider cost.
  • config/examples/gittensory.full.yml:704-705 — extend the "compute everything but take no public action" line to explicitly include AI/LLM cost.

Expected Outcome

A maintainer reading any of the three surfaces above can no longer believe agentDryRun is cost-free. No behavior change — copy only.

Links & Resources

  • Evidence: src/types.ts:1091-1093; src/settings/agent-execution.ts:38-41; src/queue/ai-review-orchestration.ts:405-411; src/queue/slop-detection.ts:65; src/queue/processors.ts:6606,6889,7101; src/github/client.ts:693 (the only place that correctly suppresses on both paused and dry_run — but that's the write layer, not the spend layer)
  • Prior related incident: gate.enabled: false does NOT stop AI spend (project memory, real 2026-07-09 incident)

Part of #5270 (roadmap epic).

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions