Skip to content

eval: versioned golden replay corpus gating every gate/prompt/model/config change #8832

Description

@JSONbored

Context

Parent: #8828 (Phase 2 — regression gate). ORB's decision policy ships without an evaluation gate, in a repo that enforces 99% patch coverage on everything else. The existing threshold/logic backtests (#8082 epic) replay rule changes; this extends that into a versioned golden corpus gating every gate-affecting change.

Requirements

Test Coverage Requirements

99%+ patch coverage, branch-counted, on the runner and corpus loader; the corpus format gets a schema validation test.

Expected Outcome

No gate-affecting change ships without demonstrating it does not reintroduce a known failure — the same discipline the repo already applies to code, applied to judgment.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions