Skip to content

fix(review): close repeated ready<->draft cycling as review evasion - #3962

Merged
JSONbored merged 3 commits into
mainfrom
claude/agitated-rhodes-61fbcd
Jul 7, 2026
Merged

fix(review): close repeated ready<->draft cycling as review evasion#3962
JSONbored merged 3 commits into
mainfrom
claude/agitated-rhodes-61fbcd

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • A contributor can currently cycle a PR between ready and draft repeatedly to harvest free AI-review/CI feedback while dodging the one-shot disposition. The two existing draft guards only fire (a) while a review pass is actively running, or (b) after a gate failure was already recorded for the current head -- neither covers a fast conversion before either of those states exists (e.g. drafting immediately on open, or before CI settles). Confirmed live on real, currently-open contributor PRs across all three of our repos.
  • Adds a new counter (pull_requests.draft_conversion_count, migration 0118) incremented on every converted_to_draft webhook, independent of head SHA. The second and every later conversion for the same PR is now enforced via the existing close + label + moderation-strike mechanism, reusing the same reviewEvasionProtection setting and audit-event family as the two existing guards.
  • Enables reviewEvasionProtection: close in .gittensory.yml (and its bundled fallback copy) for this repo -- previously off, which also meant the two existing draft/self-close evasion guards were inert here. The same setting is now enabled for metagraphed and awesome-claude via a direct config-as-code commit to each (.gittensory.yml), no code change needed there.
  • Also fixes a pre-existing, unrelated drift: an automated recalibration commit (fix(ops): recalibrate PR quality gates and slop bands #3939) updated the real .gittensory.yml's readiness.minScore but not its bundled TS fallback, which was failing the manifest-drift test on main.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves -- no tracking issue exists; this is a direct owner fix for an actively-exploited gap, linkedIssuePolicy here is preferred not required.

Validation

  • git diff --check
  • npm run typecheck
  • npm run test:coverage locally -- 100% branch coverage on every new line (repository function + queue handler), verified via lcov branch/line inspection before and after.
  • npm run db:migrations:check / npm run db:schema-drift:check
  • npm run test:ci (full local gate) -- green.
  • npm audit --audit-level=moderate -- 0 vulnerabilities.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • No auth/cookie/CORS/session changes.
  • No API/OpenAPI/MCP surface changed.
  • No UI changes.

Notes

  • gittensory's own repo currently has agent_paused = true at the DB level, which gates ALL autonomous contributor-facing actions (this guard included) independent of reviewEvasionProtection. This fix is inert here until that pause is lifted; it is already live for metagraphed and awesome-claude, whose agents are not paused.

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 7, 2026
@loopover-orb

loopover-orb Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-07 08:20:35 UTC

8 files · 1 AI reviewer · 2 blockers · readiness 93/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

Review summary
Adds a third, count-based review-evasion guard that closes a PR on its 2nd+ ready→draft conversion, independent of head SHA, mirroring the existing active-review and gate-failure guards' actuation-lock/audit/freshness-recheck pattern closely. The counter increments unconditionally on every converted_to_draft webhook via bumpPullRequestDraftConversionCount (schema.ts + migration 0118 are in parity, simple ADD COLUMN with a NOT NULL DEFAULT 0), and the guard itself correctly no-ops on the first conversion and only enforces from count>=2 with proper author/permission/freshness/autonomy gating. The slop.test.ts band-threshold rewrites look like a legitimate fix for pre-existing test drift from the unrelated #3939 recalibration, not new logic from this PR.

Nits — 5 non-blocking
  • The bump happens unconditionally per converted_to_draft webhook with no head-SHA scoping (src/db/repositories.ts bumpPullRequestDraftConversionCount) — confirm this handler already sits behind the engine's webhook-delivery dedup/idempotency check upstream in processGitHubWebhook, since a GitHub redelivery/retry of the same event would otherwise inflate the count and could trigger a false-positive close.
  • processors.ts:11912 is flagged at nesting depth 5 by the size linter; the long early-return chain in closeRepeatedDraftCyclingIfDetected (autonomy → dry-run → paused → permission → freshness) could be flattened slightly for readability, though it's consistent with the sibling guards' existing style.
  • The 'third (and every later) conversion is enforced too' test (test/unit/queue.test.ts) mocks PATCH /pulls/42 to return 200 without actually closing the PR, which lets a 3rd converted_to_draft event reach the handler in a way that wouldn't happen in production (the PR would already be closed after the 2nd enforcement) — fine as a unit test of the counter logic in isolation but worth a comment noting it's testing counter behavior, not a realistic end-to-end sequence.
  • bumpPullRequestDraftConversionCount fails open (returns 0, no audit trail) on any DB error, meaning a transient write failure silently skips detection for that cycle with no record it happened — acceptable given the documented fail-safe rationale, but consider a debug-level audit event for visibility if this proves to recur.
  • Verify (or point to) the existing webhook dedup guard earlier in processGitHubWebhook that prevents a replayed converted_to_draft delivery from double-incrementing draft_conversion_count.

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 51 registered-repo PR(s), 43 merged, 343 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 51 PR(s), 343 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 51 PR(s), 343 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@317137c). Learn more about missing BASE report.
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3962   +/-   ##
=======================================
  Coverage        ?   93.70%           
=======================================
  Files           ?      373           
  Lines           ?    35010           
  Branches        ?    12820           
=======================================
  Hits            ?    32806           
  Misses          ?     1584           
  Partials        ?      620           
Files with missing lines Coverage Δ
src/config/gittensory-repo-focus-manifest.ts 100.00% <ø> (ø)
src/db/repositories.ts 96.62% <100.00%> (ø)
src/db/schema.ts 71.18% <ø> (ø)
src/queue/processors.ts 94.65% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 7, 2026
JSONbored added 3 commits July 7, 2026 01:06
Enable reviewEvasionProtection for our own repos and add a new guard
that fires on the second (and every later) draft conversion for a PR,
independent of whether an active review or a recorded gate failure is
present. The two existing draft guards only catch dodging DURING a
still-running review or AFTER a stored gate block — a contributor who
drafts before either fires (e.g. immediately on open, or before CI
settles) can otherwise cycle ready<->draft indefinitely to harvest free
AI-review/CI feedback while sidestepping the one-shot disposition.
#3939 raised the elevated-band floor from 25 to 31, so two stacked
weak signals (30) now land in low, not elevated. Five fixtures still
asserted the pre-recalibration boundary; adjust the genuine multi-
signal cases to a real 3-signal stack (45) and the single-signal
cases to their now-correct low band, per the recalibration's own
stated design (elevated requires >=45 or 3x weak).
Rebasing onto #3938's own new settings: block (linkedIssueLabelPropagation)
left two top-level settings: keys in .gittensory.yml and its bundled
fallback -- invalid YAML that would have silently dropped one of the
two setting groups depending on parser behavior. Merge into one block.
@JSONbored
JSONbored force-pushed the claude/agitated-rhodes-61fbcd branch from bb1fbcd to 74d45b4 Compare July 7, 2026 08:09
@JSONbored
JSONbored merged commit 413705c into main Jul 7, 2026
11 checks passed
@JSONbored
JSONbored deleted the claude/agitated-rhodes-61fbcd branch July 7, 2026 08:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

1 participant