Skip to content

fix(review): production reliability pass (budget/loop/merge/notify) - #1084

Merged
JSONbored merged 6 commits into
mainfrom
fix/cache-installation-token
Jun 23, 2026
Merged

fix(review): production reliability pass (budget/loop/merge/notify)#1084
JSONbored merged 6 commits into
mainfrom
fix/cache-installation-token

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Fixes the deploy/branch split that left production running old main (10k free-tier budget, old gate) while these fixes sat on a branch.

What this lands on main (what Workers Builds deploys):

  • AI budget fail-safe — defaults to the safe max when the env var is unset/empty/NaN. Production was reading the 10k free-tier default off main and starving every dual-AI review into quota_exceeded (the bot couldn't review at all). Explicit values (incl. 0 to disable) still win.
  • Review-loop fix — refresh reviewDecision live so the planner stops re-posting a review every cycle.
  • CI gate — only branch-protection REQUIRED red checks block; non-required reds surfaced, not blocking.
  • Terminal-merge guard — classify perm/conflict/required-absent merge failures and stop retrying forever; transient retries to a cap (+ migration 0052, already applied to remote D1).
  • secret_leak dial-back so config/workflow PRs aren't false-blocked.
  • Live stats overlay for homepage counters.
  • Per-repo Discord notifications on merge/close/manual.
  • Rebase-before-review when behind base.

Follow-up PR: the full reviewbot-gate re-alignment (one-shot per head, close-on-red-CI for contributors, event-driven trigger) per the canonical non-content-gate.ts model.

All 3399 unit tests pass; tsc clean.

…at dead-lettered reviews

ROOT CAUSE of the dead-lettered reviews + stale head SHAs: createInstallationToken minted a FRESH GitHub token
on EVERY call (no cache). The converged review path mints several tokens per review (live CI aggregate, mergeable
fetch, file resolve, visual capture, permission check) and across the sweep + re-reviews this exhausted the
hourly GitHub rate limit (observed min_remaining=0 at 6753 calls/hr) → reviews errored → dead-lettered → missed
synchronize webhooks → stale stored head SHAs → reviews ran against the old (green) commit → wrong 'CI is green'
approvals after a rebase. FIX: in-isolate token cache (valid ~1h, 2-min safety margin) → ~1 mint/hour/installation
instead of per-call. Tests: clear the cache per-test; token-count assertions drop to the cached count.
…s on idle PRs)

The sweep re-gated + ran auto-maintain but never re-published the unified comment, so an idle PR kept a stale
comment ('safe to merge' from before a fix) + stale status label forever even after the head was re-synced.
Switch to reReviewStoredPullRequest (rebuild advisory → re-publish comment with current head/CI → re-run
auto-maintain) so comment + label + action reflect reality. Paced at SWEEP_MAX_PRS/sweep; cheap with token caching.
…rise) + interactive re-run checkbox

The 10k AI_DAILY_NEURON_BUDGET default starved EVERY dual-AI review (~573 neurons each)
into quota_exceeded on the enterprise Workers AI plan (account does hundreds of k/day),
so no review ever produced its synthesis. Models were correct all along (gpt-oss-120b +
nemotron-3-120b-a12b); the self-imposed cap was the sole blocker.

- AI_DAILY_NEURON_BUDGET 10k -> 2M; code clamp ceiling 1M -> 10M (runaway backstop, not a cap)
- AI_MAX_OUTPUT_TOKENS 256 -> 1024 (room for a real assessment + suggestions + risks)
- unified-comment: render the re-run checkbox OUTSIDE the alert blockquote — GitHub disables
  task-list checkboxes inside a blockquote, so clicking it never fired issue_comment.edited
- sweep: keep the full re-review re-publish (affordable under the raised budget) so idle PRs
  refresh their comment/label/synthesis and dead-lettered reviews get a fresh verdict
… review (no comment/audit/close)

THE root cause of 'nothing works': formatGateCheckOutput returned output.title unbounded.
GitHub 422s an over-long check-run title; the throw escaped createOrUpdateGateCheckRun
(unwrapped) and the outer catch RE-THREW, so the review aborted before publishing the
comment, writing the audit row, and running auto-maintain (merge/close). #1075 appended
failing-check names to the title, so RED-CI PRs got long titles → 422 → they were never
reviewed or closed — exactly the PRs that should auto-close.

- advisory.ts: cap gate output.title at 255 (matches the annotation cap) — prevents the 422
- processors.ts: wrap createOrUpdateGateCheckRun in a local try/catch — a check-run API
  failure now finalizes the pending check to neutral and CONTINUES (records
  github_app.gate_check_failed_nonfatal) instead of dead-lettering the whole review
- tests updated for the non-fatal-continue behavior
- ai-review: neuron budget defaults to the safe max when unset/empty/NaN. The deployed worker read the 10k free-tier default off main and starved every dual-AI review into quota_exceeded; an explicit value (incl. "0" to disable) still wins.
- review loop: refresh reviewDecision live so the action planner does not re-post a review every cycle.
- CI gate: only branch-protection REQUIRED red checks block; non-required reds are surfaced, not blocking.
- merges: classify terminal merge failures (perm/conflict/required-absent) and stop retrying them every sweep; transient failures retry to a cap (+ migration 0052).
- safety: dial back the secret_leak heuristic so config/workflow PRs are not false-blocked.
- stats: read the live audit_events overlay for homepage counters.
- notify: per-repo Discord embeds on merge/close/manual.
- rebase: update-branch before review when behind base.
@ghost

ghost commented Jun 23, 2026

Copy link
Copy Markdown

Caution

🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

🛑 Gittensory review — closed

26 files · 1 AI reviewers · 1 blocker · readiness 93/100 · CI failing · unstable

🛑 Closed — Possible leaked secret in the diff (seed_or_mnemonic, bittensor_key) — Remove the secret from the diff, rotate the exposed credential, then re-run the gate.

Why this is blocked

  • Possible leaked secret in the diff (seed_or_mnemonic, bittensor_key) — Remove the secret from the diff, rotate the exposed credential, then re-run the gate.

CI checks failing

  • Gittensory Gate — Gittensory Gate: Possible leaked secret in the diff (seed_or_mnemonic, bittensor_key)
  • codecov/patch — 42.96% of diff hit (target 95.12%)
  • codecov/project — 94.75% (-0.38%) compared to 38c6174
Signal Result Evidence
Code review ❌ 1 blocker 1 reviewers, synthesized
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Review load ✅ 20/20 Readiness component derived from cached public PR metadata and labels; size label size:XL.
Validation evidence ✅ 25/25 PR body includes validation/test evidence.
Open PR queue ✅ 10/10 0 open PR(s), 0 likely reviewable.
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 85 PR(s), 250 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Nits — 1 non-blocking
  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 85 PR(s), 250 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Review load = cached public PR metadata such as size labels, changed paths, and preflight status.
  • Open PR queue = repo-wide review pressure; it is not a PR quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
  • Re-run Gittensory review

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

@ghost ghost added gittensory:reviewed gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. labels Jun 23, 2026

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory requests changes — CI is not green:

  • CI is failing (Gittensory Gate)
  • Possible leaked secret in the diff (seed_or_mnemonic, bittensor_key)

@JSONbored JSONbored self-assigned this Jun 23, 2026

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory requests changes — CI is not green:

  • CI is failing (Gittensory Gate)
  • Possible leaked secret in the diff (seed_or_mnemonic, bittensor_key)

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory requests changes — CI is not green:

  • CI is failing (Gittensory Gate)
  • Possible leaked secret in the diff (seed_or_mnemonic, bittensor_key)

…ors + public-stats)

Resolved: processors.ts kept the live CI aggregate + adopted main's changedPathsForGuardrail; public-stats.ts took main's #1059 rewrite (RC8 live-overlay to be re-applied cleanly as a follow-up). 3424 tests pass.
@dosubot dosubot Bot added the size:XL label Jun 23, 2026
@codecov

codecov Bot commented Jun 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 42.96296% with 77 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.75%. Comparing base (38c6174) to head (b465c8c).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/github/backfill.ts 20.68% 15 Missing and 8 partials ⚠️
src/queue/processors.ts 42.85% 8 Missing and 8 partials ⚠️
src/services/notify-discord.ts 35.00% 11 Missing and 2 partials ⚠️
src/services/agent-action-executor.ts 57.89% 5 Missing and 3 partials ⚠️
src/services/merge-failure.ts 46.15% 1 Missing and 6 partials ⚠️
src/github/pr-actions.ts 0.00% 5 Missing ⚠️
src/db/repositories.ts 33.33% 4 Missing ⚠️
src/settings/agent-actions.ts 50.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1084      +/-   ##
==========================================
- Coverage   95.12%   94.75%   -0.38%     
==========================================
  Files         151      153       +2     
  Lines       18353    18465     +112     
  Branches     6638     6683      +45     
==========================================
+ Hits        17459    17497      +38     
- Misses        366      416      +50     
- Partials      528      552      +24     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory requests changes — CI is not green:

  • CI is failing (Gittensory Gate, codecov/patch, codecov/project)
  • Possible leaked secret in the diff (seed_or_mnemonic, bittensor_key)

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory requests changes — CI is not green:

  • CI is failing (Gittensory Gate, codecov/patch, codecov/project)
  • Possible leaked secret in the diff (seed_or_mnemonic, bittensor_key)

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory requests changes — CI is not green:

  • CI is failing (Gittensory Gate, codecov/patch, codecov/project)
  • Possible leaked secret in the diff (seed_or_mnemonic, bittensor_key)

@JSONbored
JSONbored merged commit 5f76fae into main Jun 23, 2026
16 of 19 checks passed
@JSONbored
JSONbored deleted the fix/cache-installation-token branch June 23, 2026 07:04

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory requests changes — CI is not green:

  • CI is failing (Gittensory Gate, codecov/patch, codecov/project)
  • Possible leaked secret in the diff (seed_or_mnemonic, bittensor_key)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant