Skip to content

fix(queue): owner-verified release for per-PR actuation lock - #3057

Closed
andriypolanski wants to merge 1 commit into
JSONbored:mainfrom
andriypolanski:fix/pr-actuation-lock-stale-release-race
Closed

fix(queue): owner-verified release for per-PR actuation lock#3057
andriypolanski wants to merge 1 commit into
JSONbored:mainfrom
andriypolanski:fix/pr-actuation-lock-stale-release-race

Conversation

@andriypolanski

@andriypolanski andriypolanski commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Summary

The shared per-PR actuation mutex (claimPrActuationLock / releasePrActuationLock) used a constant lock value and unconditional del on release. If a holder ran past the 600s TTL, a later worker could claim the lock; when the first worker’s finally ran, it deleted the second worker’s live lock — reopening the exact race the mutex exists to prevent (concurrent merge/close/reopen-reclose on the same PR).

This PR stores a per-holder ownership token on claim and releases via atomic compare-and-delete (releaseIfValue). A stale holder’s finally no longer deletes a later claimer’s lock.

Changed files

File Change
src/env.d.ts Add optional releaseIfValue(key, value) to SELFHOST_TRANSIENT_CACHE — atomic delete only when the stored owner matches.
src/selfhost/redis-cache.ts Implement releaseIfValue with a Lua compare-and-delete script.
src/queue/processors.ts claimPrActuationLock returns PrActuationLockClaim (token / fail-open "" / contended null); releasePrActuationLock takes the token and calls releaseIfValue. Add claimTransientLockWithOwnerToken and releaseTransientLockIfOwner helpers. Update three call sites (maybeRunAgentMaintenance, maybeCloseDraftDodgeAttempt, maybeRecloseDisallowedReopen). Remove the documented KNOWN LIMITATION.
test/helpers/d1.ts Implement releaseIfValue on the in-memory transient cache used by unit tests.
test/unit/queue.test.ts Update lock tests for the new return type; add regression test that a stale holder’s release does not delete a later claimer’s lock.
test/unit/selfhost-redis-cache.test.ts Unit tests for releaseIfValue match / mismatch paths.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked an issue, or this is small enough that the summary explains why an issue is not needed.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

  • Full gate run via npm run test:ci green locally (~7 min, 8,234+ tests). Required npm run selfhost:env-reference first on this tree (stale line refs in selfhost-env-reference.ts unrelated to this PR — do not commit unless your branch also needs it).
  • codecov/patch not verified locally — depends on GitHub CI upload.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
  • API/OpenAPI/MCP behavior is updated and tested where needed.
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
  • Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

No UI, API, OpenAPI, or MCP surface changes — internal queue concurrency fix only.

UI Evidence

N/A — no visible UI changes.

Test plan

  • releaseIfValue — deletes when owner matches, refuses when owner differs (selfhost-redis-cache.test.ts)
  • Stale-holder regression — stale token release leaves later holder’s lock intact (queue.test.ts)
  • Existing claimPrActuationLock suite updated for token return type (claim / contended / fail-open / concurrent)
  • Draft-dodge and reopen-reclose lock-contention integration paths still pass (queue.test.ts)
  • Full npm run test:ci green (8,234+ tests)

Notes

PrActuationLockClaim contract:

  • non-empty string → lock claimed; pass to releasePrActuationLock in finally
  • "" → fail-open (no claim() primitive or claim threw); skip release
  • null → contended; skip work (or throw PrActuationLockContendedError on retryable paths)

When releaseIfValue is unavailable on a cache adapter, release is a no-op and the key expires via TTL rather than risking an unconditional delete.

Analogue: claimRegateFanoutSlot() uses the same atomic conditional-update pattern for burst dedup.

@andriypolanski
andriypolanski requested a review from JSONbored as a code owner July 4, 2026 10:46
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.11%. Comparing base (b791587) to head (252a3fb).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3057   +/-   ##
=======================================
  Coverage   96.11%   96.11%           
=======================================
  Files         263      263           
  Lines       28960    28975   +15     
  Branches    10537    10541    +4     
=======================================
+ Hits        27834    27849   +15     
  Misses        492      492           
  Partials      634      634           
Files with missing lines Coverage Δ
src/queue/processors.ts 92.96% <100.00%> (+0.03%) ⬆️
src/selfhost/redis-cache.ts 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 4, 2026
@loopover-orb

loopover-orb Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Caution

🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

🛑 Gittensory review result - reject/close recommended

Review updated: 2026-07-04 18:07:30 UTC

6 files · 1 AI reviewer · 1 blocker · readiness 73/100 · CI green · clean

🛑 Suggested Action - Reject/Close

  • AI reviewers agree on a likely critical defect: src/queue/processors.ts:3488 makes claimTransientLockWithOwnerToken take the lock whenever claim() exists, but src/queue/processors.ts:3502 then refuses to release it when releaseIfValue() is absent, so any existing adapter with claim() but not releaseIfValue() will leave the PR actuation mutex held for PR_ACTUATION_LOCK_TTL_SECONDS after a successful pass and cause later maybeRunAgentMaintenance/maybeCloseDraftDodgeAttempt/maybeRecloseDisallowedReopen work for the same PR to defer or throw until the TTL expires
  • change the claim helper to fail open unless both primitives are present, for example `if (!env.SELFHOST_TRANSIENT_CACHE?.claim || !env.SELFHOST_TRANSIENT_CACHE.releaseIfValue) return ""
  • `, or make releaseIfValue mandatory everywhere this cache contract is implemented. — Resolve the flagged defect, or override if the AI reviewers are mistaken, then re-run the gate.

Review summary
The change correctly moves the per-PR actuation lock from an unconditional delete to an owner-token compare-and-delete path, and the Redis implementation uses the right atomic Lua shape. The main remaining risk is the compatibility path: a cache adapter that supports atomic claim but has not added releaseIfValue will now hold every successful PR actuation lock until the 600s TTL, which is a reachable queue/gate delay because releasePrActuationLock intentionally no-ops in that case. The regression coverage for stale-owner release is meaningful, but it also locks in that TTL-only behavior for partially-upgraded adapters.

Blockers

  • src/queue/processors.ts:3488 makes claimTransientLockWithOwnerToken take the lock whenever claim() exists, but src/queue/processors.ts:3502 then refuses to release it when releaseIfValue() is absent, so any existing adapter with claim() but not releaseIfValue() will leave the PR actuation mutex held for PR_ACTUATION_LOCK_TTL_SECONDS after a successful pass and cause later maybeRunAgentMaintenance/maybeCloseDraftDodgeAttempt/maybeRecloseDisallowedReopen work for the same PR to defer or throw until the TTL expires; change the claim helper to fail open unless both primitives are present, for example `if (!env.SELFHOST_TRANSIENT_CACHE?.claim || !env.SELFHOST_TRANSIENT_CACHE.releaseIfValue) return "";`, or make releaseIfValue mandatory everywhere this cache contract is implemented.
Nits — 6 non-blocking
  • test/unit/queue.test.ts:5557 uses SELFHOST_TRANSIENT_CACHE.set() to overwrite the held key instead of driving the later-holder step through the same claim path after TTL expiry, so the test proves compare-delete semantics but not the full stale-holder/later-claimer production sequence.
  • src/env.d.ts:37 still documents claim() as falling back to a non-atomic get/set pair when absent, but this file’s current lock path now fails open for missing claim() and introduces releaseIfValue(); update the contract comment so adapter authors do not preserve the older behavior by mistake.
  • src/queue/processors.ts:3488 should treat `claim` and `releaseIfValue` as one capability for owner-token locks, or the partial-adapter path should be explicitly bounded to a much shorter TTL with a comment explaining why the 600s actuation lock delay is acceptable.
  • test/unit/queue.test.ts:5557 should simulate expiry by removing the stale key and then acquiring the later lock through `claimPrActuationLock`, so the regression covers the real owner-token producer instead of a hand-written payload.
  • src/env.d.ts:37 should describe the resolved cache contract as `claim + releaseIfValue` for exclusive owner-token locks and fail-open when the pair is unavailable.
  • Readiness score is below the configured threshold — Use the readiness panel as advisory maintainer context; the score does not block this PR.

Why this is blocked

  • src/queue/processors.ts:3488 makes claimTransientLockWithOwnerToken take the lock whenever claim() exists, but src/queue/processors.ts:3502 then refuses to release it when releaseIfValue() is absent, so any existing adapter with claim() but not releaseIfValue() will leave the PR actuation mutex held for PR_ACTUATION_LOCK_TTL_SECONDS after a successful pass and cause later maybeRunAgentMaintenance/maybeCloseDraftDodgeAttempt/maybeRecloseDisallowedReopen work for the same PR to defer or throw until the TTL expires; change the claim helper to fail open unless both primitives are present, for example `if (!env.SELFHOST_TRANSIENT_CACHE?.claim || !env.SELFHOST_TRANSIENT_CACHE.releaseIfValue) return "";`, or make releaseIfValue mandatory everywhere this cache contract is implemented.
Signal Result Evidence
Code review ❌ 1 blocker 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ❌ 5/25 Preflight is holding this PR: the review lane is unavailable, so it is not ready for automated review.
Contributor workload ✅ 10/10 Author activity: 107 registered-repo PR(s), 70 merged, 5 issue(s).
Contributor context ✅ Confirmed Gittensor contributor andriypolanski; Gittensor profile; 107 PR(s), 5 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Review context
  • Author: andriypolanski
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: TypeScript, JavaScript, Python, Rust, Cuda, Kotlin, MDX, Scala
  • Official Gittensor activity: 107 PR(s), 5 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Explain no-issue PR.
  • Await review-lane availability.
  • Refresh registry data or choose a registered active repo.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb

loopover-orb Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Gittensory is closing this pull request on the maintainer's behalf (AI reviewers agree on a likely critical defect: src/queue/processors.ts:3488 makes claimTransientLockWithOwnerToken take the lock whenever claim() exists, but src/queue/processors.ts:3502 then refuses to release it when releaseIfValue() is absent, so any existing adapter with claim() but not releaseIfValue() will leave the PR actuation mutex held for PR_ACTUATION_LOCK_TTL_SECONDS after a successful pass and cause later maybeRunAgentMaintenance/maybeCloseDraftDodgeAttempt/maybeRecloseDisallowedReopen work for the same PR to defer or throw until the TTL expires; change the claim helper to fail open unless both primitives are present, for example `if (!env.SELFHOST_TRANSIENT_CACHE?.claim || !env.SELFHOST_TRANSIENT_CACHE.releaseIfValue) return "";`, or make releaseIfValue mandatory everywhere this cache contract is implemented.). This is an automated maintenance action — to pursue this change, please open a new pull request with the issues resolved. Closed PRs may be analyzed later to improve review accuracy, but they are not automatically reopened or re-reviewed.

@loopover-orb loopover-orb Bot closed this Jul 4, 2026
@andriypolanski
andriypolanski deleted the fix/pr-actuation-lock-stale-release-race branch July 16, 2026 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant