chore(review): enable + validate E2E test generation end-to-end

[JSONbored]

Part of #4189. Depends on #4190, #4191, #4192, #4193, #4194, #4195, #4196 (the full v1 comment-only pipeline: config, generation core, gate, render, dispatch, command, auto-trigger). #4197 (delivery escalation) and #4198 (self-host execution) validate separately once they ship, following the same discipline below.

Context

Once config, generation, gating, rendering, dispatch, the PR command, and the auto-trigger all land, the feature still needs to actually be turned on and verified against real PR traffic before it's trustworthy — mirroring the exact role #3611 played for the visual-capture epic (#3607): building the pieces and turning them on are different milestones, and this repo's own convention treats "enable + validate" as its own tracked issue rather than an implicit side effect of the last engineering PR.

Requirements

• Enable GITTENSORY_REVIEW_E2E_TESTS (the global kill-switch from feat(review): register e2eTests as the sixth converged-feature key #4190) on a real self-host or hosted deployment, and turn on features.e2eTests for at least one real repo via its .gittensory.yml.
• Open a real (or deliberately test) PR that's missing test coverage against that repo and confirm: the manifest_missing_tests auto-trigger (feat(review): auto-trigger E2E test generation on the manifest_missing_tests signal #4196) fires correctly, the generated Playwright test block renders correctly in the unified comment (feat(review): render + comment-assembly for generated E2E test blocks #4193), the generated test source is plausible and relevant to the actual diff (a human-reviewed spot check, not just "did it parse"), and the explicit @gittensory generate-tests command (feat(commands): wire @gittensory generate-tests PR-comment command #4195) also works standalone on a PR where the signal didn't fire.
• Confirm the cache-miss-only discipline (feat(review): dispatch wiring for e2eTests in queue/processors.ts #4194) holds in practice: pushing a new commit to the same PR re-triggers generation exactly once per fresh pass, and re-rendering a cached pass does not re-invoke the LLM (verify via actual cost/audit-event logs, not just unit tests).
• Confirm the feature is fully fail-safe under real conditions: an LLM failure, a malformed model response, or a rate-limit must degrade to "no block emitted" and never hold or block an otherwise-mergeable PR — this is a design invariant carried over from every other converged feature and from fix-handoff specifically; verify it holds in practice, not just in tests.
• Monitor cost (LLM spend, BYOK vs. default model split) and latency impact for at least a week of real traffic before calling v1 done — this is a real, per-PR LLM cost driver, worth the same production observation window the visual-capture epic required for its own most-expensive-step (Browser Rendering).
• Document a rollback plan (flip GITTENSORY_REVIEW_E2E_TESTS back off, or unset the per-repo override) if something goes wrong.

Deliverables

• Flag enabled on at least one real deployment and repo.
• At least one real PR with a verified, human-reviewed-correct generated test block, via both the auto-trigger and the explicit command paths.
• Confirmed cache-miss-only behavior against real audit/cost logs.
• Confirmed fail-safe degradation under at least one induced failure (bad model response, timeout, or rate-limit).
• A documented rollback plan.
• A week of production cost/latency observation with no regressions before calling v1 shipped.

Expected outcome

A real gittensory-reviewed PR that's missing test coverage gets a correct, human-verified AI-generated Playwright test suggestion with no contributor or maintainer action required — the epic's v1 scope, fully live and observed under real traffic, not just passing unit tests.

Resources / examples

• #3611 ("Enable + validate automated visual capture end-to-end for metagraphed") — the exact precedent for this issue's shape and rigor bar, including its own week-long production-observation requirement.

Effort

M — mostly verification/observation once #4190-#4196 land, not new engineering.

[linear-code]

JSO-3231

[JSONbored]

Status: everything this issue depends on (#4190-#4196) is merged. The feature is code-complete, fully tested, and dormant behind flags — nothing is enabled anywhere yet.

Readiness summary

• Global kill-switch: `GITTENSORY_REVIEW_E2E_TESTS` (wrangler var) — currently `false` everywhere. Setting it `true` on a deployment does nothing by itself; it only lifts the operator-level gate.
• Per-repo opt-in: `features.e2eTests: true` in that repo's `.gittensory.yml` — required IN ADDITION to the global flag (both must be true, config-as-code precedence: yml > DB > default).
• Delivery mode: defaults to `comment` (posts the generated test as a suggestion in a reply comment). Setting `review.e2e_test_delivery: commit` in the same `.gittensory.yml` escalates to pushing it as a real commit onto the PR's own head branch — this is unconditionally blocked for a confirmed Gittensor miner's PR regardless of this setting (fix(scoring): exclude maintainer-generated test commits from contributor scoring #4201's scoring-integrity safeguard), so it's safe to enable for any repo, miner-authored PRs or not.
• Two ways generation fires: the explicit `@gittensory generate-tests` comment (maintainer/collaborator/confirmed_miner only), and automatically when the existing `manifest_missing_tests` signal fires (requires `review.test_expectations` configured + `manifestPolicyGateMode` not `off`, since that's the only place the signal is computed) — both share one code path (`runE2eTestGenerationAndDeliver`) and one daily neuron budget, so there's no separate cost surface to reason about.

Rollback plan

Flip `GITTENSORY_REVIEW_E2E_TESTS` back to `false` (or unset the per-repo `features.e2eTests` override) — byte-identical to today, no persisted state to unwind. A repo that already has generated test files sitting on old PR branches keeps them; nothing auto-reverts committed test files, since they're just regular commits at that point.

What's left (this issue's actual deliverables)

All of these need a real deployment + real PR traffic + your own judgment call, so I haven't done them autonomously:

• Flip the flag on for one real repo.
• A real PR, missing tests, where you spot-check the generated Playwright test is actually plausible/relevant (not just "did it parse").
• Confirm the explicit command works standalone too, on a PR where the auto-trigger signal didn't fire.
• Watch audit/cost logs to confirm the cache-miss-only discipline holds under real traffic (unit tests already cover this, but this issue asks for a real-traffic check).
• Induce one real failure (bad model response / timeout / rate-limit) and confirm it degrades to no-block, never holds a mergeable PR.
• A week of cost/latency observation before calling v1 done.

Ping me (or just flip the flag) whenever you want to start the observation window — happy to help watch the logs once it's live.