Skip to content

feat(review): push generated E2E tests as a real PR-branch commit - #4245

Merged
JSONbored merged 1 commit into
mainfrom
feat/e2e-test-gen-commit-delivery-4197-4201
Jul 8, 2026
Merged

feat(review): push generated E2E tests as a real PR-branch commit#4245
JSONbored merged 1 commit into
mainfrom
feat/e2e-test-gen-commit-delivery-4197-4201

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • Adds an opt-in review.e2e_test_delivery: commit config mode so @gittensory generate-tests can push the AI-generated Playwright test onto the PR's own head branch as a real commit, instead of only replying with a suggestion comment.
  • Adds a scoring-integrity safeguard: commit delivery is blocked outright for a PR authored by a confirmed Gittensor miner, since gittensory never computes the authoritative score itself (an external validator reads the merged PR straight from GitHub) — a maintainer-authored commit landing on a miner's branch would be indistinguishable from a line the miner wrote, inflating their apparent contribution. This check runs unconditionally whenever commit delivery is configured, regardless of the repo's own settings.
  • Threads the repo's agentPaused/agentDryRun action mode through @gittensory generate-tests (previously unchecked, unlike every other action command).
  • Every non-committed outcome (declined for a GitHub-side reason, blocked by the miner safeguard, or plain comment-only mode) still renders the generated test as a suggestion in the reply comment, so the generated content is never silently dropped.

Closes #4197
Closes #4201

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally (unsharded) — 100% branch coverage on every changed/new line in this diff, confirmed via raw lcov BRDA data, not just the summary table.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries — including the miner-safeguard branch, the not-found/no-author-login fallback, the GitHub-write-declined path, the unexpected-failure-mapped-to-declined path, and the dry-run/paused gate.

Also ran (beyond the checklist above, since this repo's test:ci covers more than the template lists): npm run db:migrations:check, npm run db:schema-drift:check, npm run selfhost:env-reference:check, npm run selfhost:validate-observability, npm run cf-typegen:check, npm run test:engine-parity, the @jsonbored/gittensory-engine package's own test suite, npm run build:miner, npm run test:miner-pack, npm run rees:test, npm run ui:openapi:settings-parity, npm run ui:version-audit, npm run docs:drift-check, npm run manifest:drift-check, npm run command-reference:check, npm run ui:test.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests (no write access / fork without maintainer edits, and the branch-moved-since-this-pass-started race, both covered).
  • API/OpenAPI/MCP behavior is updated and tested where needed (no OpenAPI/MCP surface changed by this PR).
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks (no UI changed).
  • UI Evidence — not applicable, no visible UI/frontend/docs/extension change in this PR.
  • Public docs/changelogs are updated where needed (.gittensory.yml.example and config/examples/gittensory.full.yml document the new review.e2e_test_delivery field); changelog itself is untouched.

Notes

, #4201)

Adds an opt-in review.e2e_test_delivery: commit mode so @gittensory
generate-tests pushes the generated Playwright test onto the PR's own
head branch instead of only replying with a suggestion comment. Reuses
the git/trees -> git/commits -> git/refs write pattern from
repo-doc-pr.ts, but PATCHes an existing ref instead of creating one.

Delivery is blocked for a confirmed Gittensor miner's PR (#4201): since
gittensory never computes the authoritative score itself (an external
validator reads the merged PR from GitHub), a maintainer-authored
commit on a miner's branch would be indistinguishable from a line the
miner wrote, inflating their apparent contribution. That check runs
regardless of the repo's delivery config.

Also threads the repo's own agentPaused/agentDryRun action mode through
the command (previously unchecked), and folds every non-committed
outcome (declined, blocked, or comment-only) back to rendering the
generated test as a suggestion, so a maintainer never loses the
content just because the heavier delivery mode didn't apply.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
gittensory-ui db239a7 Commit Preview URL

Branch Preview URL
Jul 08 2026, 06:31 PM

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.79%. Comparing base (0796aa7) to head (db239a7).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4245   +/-   ##
=======================================
  Coverage   93.78%   93.79%           
=======================================
  Files         394      395    +1     
  Lines       36628    36672   +44     
  Branches    13403    13422   +19     
=======================================
+ Hits        34351    34395   +44     
  Misses       1622     1622           
  Partials      655      655           
Files with missing lines Coverage Δ
packages/gittensory-engine/src/focus-manifest.ts 99.16% <100.00%> (+<0.01%) ⬆️
src/github/e2e-test-commit.ts 100.00% <100.00%> (ø)
src/queue/processors.ts 95.34% <100.00%> (+0.02%) ⬆️
src/review/e2e-test-gen-render.ts 100.00% <100.00%> (ø)
src/review/guardrail-config.ts 100.00% <ø> (ø)
src/signals/focus-manifest.ts 99.62% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 8, 2026
@loopover-orb

loopover-orb Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-08 18:41:37 UTC

13 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • Touches a guarded path — held for manual review: This PR changes guardrail-protected path(s): src/queue/processors.ts (matched src/queue/**), src/review/guardrail-config.ts (matched src/review/guardrail-config.ts).

Review summary
This PR adds an opt-in review.e2e_test_delivery: "commit" mode that pushes AI-generated E2E tests directly onto a PR's head branch (via git/trees→git/commits→a ref PATCH, mirroring repo-doc-pr.ts's write chokepoint), guarded by a scoring-integrity safeguard that unconditionally blocks commit delivery for confirmed Gittensor miners and by the same agentPaused/agentDryRun mode gate every other action command already respects (a real gap fix — generate-tests previously ignored it). The config field is wired at every required site (schema, overlay, JSON serialization, both example ymls, focus-manifest exhaustiveness test, signals re-export), every non-committed outcome (declined/blocked/comment-only) still renders the test as a suggestion so content is never silently dropped, and the new module is added to the engine-decision guardrail globs given its bigger blast radius (writing to a contributor's own branch). Test coverage is thorough and traces real branches: committed, miner-blocked, 403/422 declines, uncached-head decline, mapped 500 error, no-author-login ternary arm, dry-run, and agent-paused — all closing #4197/#4201 as claimed.

Nits — 7 non-blocking
  • src/github/e2e-test-commit.ts's header comment references an `isMinerAuthoredBranch` check the caller is supposed to run, but processors.ts actually uses `getCachedOfficialMinerDetection` directly — confirm that name isn't stale from an earlier draft.
  • Unlike `review.inline_comments` (which needed an operator flag + cutover allowlist on top of the per-repo toggle per its own history), `e2e_test_delivery: commit` writing real commits to a contributor's branch is gated only by the existing `GITTENSORY_REVIEW_E2E_TESTS` flag plus the per-repo config bit — worth confirming that's an intentional, lighter rollout bar for a write-capable feature.
  • src/github/e2e-test-commit.ts's 403/404/422/409 branch could name the status codes as constants for readability, per the flagged magic-number smell.
  • defaultE2eTestFilePath's fixed `e2e/gittensory-pr-{n}.spec.ts` path has no per-repo override read from config yet — acknowledged in its own comment as a deferred enhancement, so just confirm that's intentional for this first delivery mode.
  • Add a short-lived comment/name fix in e2e-test-commit.ts's header so it points at `getCachedOfficialMinerDetection`, matching what processors.ts actually calls.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #4197, #4201
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (2 linked issues).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 52 registered-repo PR(s), 43 merged, 497 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 52 PR(s), 497 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Linked issue satisfaction

Partially addressed
The PR fully implements the issue's primary ask — commit mode via git/trees→git/commits→PATCH git/refs onto the PR's own head branch, the exact 'Generated-by: gittensory (invoked by @​actor)' provenance trailer, maintainer-only gating (plus newly threading dry-run/paused modes), a post-commit confirmation comment linking the commit, and a scoring-integrity safeguard for confirmed miners — but it om

Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 52 PR(s), 497 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 8, 2026
@JSONbored
JSONbored merged commit 7b35640 into main Jul 8, 2026
12 checks passed
@JSONbored
JSONbored deleted the feat/e2e-test-gen-commit-delivery-4197-4201 branch July 8, 2026 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

1 participant