Skip to content

epic(ai): AI-review pipeline reliability, verdict fidelity and cost defects #9480

Description

@JSONbored

Summary

Tracking epic for AI-review pipeline defects found in the 2026-07-27 adversarial audit — reliability, verdict fidelity, and cost.

The single most severe AI finding (a real blocker silently collapsing to a clean pass → false merge) is filed under the wrong-verdict epic as #9460, because it belongs with the other gate-verdict defects. Everything else is here.

Sub-issues

Issue Class Why it matters
#9476claude_stalled_no_output escapes the no-retry guard Reliability / cost 3× timeout per review; parks all 8 queue slots during a Claude slowdown
#9477 — cache fingerprint omits prompt + verdict-logic version Verdict fidelity 11 policy changes shipped without invalidating cached verdicts
#9478 — Ollama fallback: no num_ctx, full authority, misattributed votes Verdict fidelity A Claude outage silently downgrades every verdict
#9479 — stdin EPIPE crash, temp-dir leak, no push debounce, budget under-books 6× Reliability / cost One of these can kill the worker

Deployment context (verified live on edge-nl-01, 2026-07-27)

How these interact

#9476 makes fallback far more frequent than it appears, which makes #9478's quality asymmetry the normal path during a Claude incident rather than an edge case. #9477 then makes any fix to either invisible for already-reviewed heads. They should be sequenced #9476#9478#9477.

Acceptance criteria

  • All four sub-issues closed.
  • A provider timeout costs one attempt, not three.
  • The serving provider is recorded in every decision record and named in public copy when it is not the primary.
  • A prompt or policy change automatically invalidates affected cache rows without a manual constant bump.

Verified solid (recorded so it is not re-audited)

The audit specifically probed and cleared: truncated-JSON handling at the 4096-token limit (extractLastJsonObject is brace-depth and string aware, returns null, degrades to an inconclusive hold — a truncated response cannot become a false merge or false close); the consensus combine branch (genuinely fail-closed, #9074's agreement check sound); AI-output→GitHub-comment sanitization (neutralizePublicMarkdown — no injection path found); inline anchoring (#9076's fixes hold; blockers must land on added lines and newline counts are preserved); subprocess credential hygiene (strict env allowlist, --tools "", --strict-mcp-config, system prompt via file not argv, redactSecrets over stderr); and the circuit breaker's lost-update fix.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.orbGittensory Orb related - maintainer self-hosting analytics.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions