feat(miner): add governor rate-limit/budget Prometheus metrics + pressure alerts - #5604
Conversation
…sure alerts Adds `gittensory-miner governor metrics` (rate-limit-remaining and cap-usage gauges) and two matching Prometheus alerts, mirroring the queue-metrics pattern shipped for #5186.
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
loopover-ui | 6df377b | Commit Preview URL Branch Preview URL |
Jul 13 2026, 09:13 AM |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5604 +/- ##
=======================================
Coverage 95.00% 95.01%
=======================================
Files 573 574 +1
Lines 45629 45684 +55
Branches 14661 14661
=======================================
+ Hits 43350 43405 +55
Misses 1528 1528
Partials 751 751
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
|
Warning 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 ⏸️ Gittensory review result - manual review recommendedReview updated: 2026-07-13 08:59:48 UTC
⏸️ Suggested Action - Manual Review Review summary Blockers
Nits — 6 non-blocking
Concerns raised — review before merging
📋 Copy for AI agents — paste into your coding agent
Linked issue satisfactionPartially addressed Review context
Contributor next steps
Signal definitions
[BETA] Chat with GittensoryAsk Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.
Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands 🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
…adjusted remaining evaluateLocalRateLimit's `remaining` field already accounts for one hypothetical next write (remaining = limit - effectiveCount - 1), so a bucket at count=2/limit=3 (one write still allowed) rendered the same 0 ratio as a fully exhausted count=3/limit=3 bucket. Recover true headroom as `allowed ? remaining + 1 : 0` before dividing by limit.
|
Fixed in 6df377b. |
Summary
gittensory-miner governor metrics, a Prometheus text-exposition endpoint for the governor's persisted state (Persist governor cross-attempt state (rate-limit, budget, convergence, reputation, self-plagiarism) #5134):gittensory_miner_governor_rate_limit_remaining_ratio{scope,action_class,repo}(gauge, headroom remaining in the current write-rate-limit window as a fraction of the configured limit) andgittensory_miner_governor_cap_usage_ratio{dimension}(gauge, cumulative budget/turns/elapsed-time usage as a fraction of the configured ceiling).gittensory-miner-governorPrometheus rule group:GittensoryMinerGovernorRateLimitPressureHigh(remaining ratio < 10%, sustained 10m) andGittensoryMinerGovernorCapUsageHigh(cap usage ratio > 90%, sustained 10m).loadRateLimitState()/loadCapUsage()with the engine's existing pure calculators (evaluateLocalRateLimit,evaluateGovernorCaps) against the same default policies the production loop already falls back to when no.gittensory-ams.ymloverride is configured (DEFAULT_WRITE_RATE_LIMIT_POLICIES,DEFAULT_AMS_POLICY_SPEC.capLimits) — no invented thresholds. Strictly read-only: it does not gate, retry, or modify governor decision logic (governor-chokepoint.js/governor-chokepoint-persisted.jsare untouched).queue metrics/portfolio-queue-alerts pattern shipped for Add a Prometheus alert rule for stuck/stale portfolio-queue claimed items #5186 (merged in feat(miner): add portfolio-queue Prometheus metrics + stuck/backlog alerts #5603): same escaping helpers, deterministic sorted output, always-emit HELP/TYPE, and a fixture test that parses the realprometheus/rules/alerts.ymland pinsexpr/for/labels/annotationsto the renderer's exported metric-name constants.Scope note
#5187 depends on #5134 (governor cross-attempt state persistence, already shipped) and says to reference "the metric/state exposed by #5134" — but #5134 only persisted state to SQLite, it never shipped a Prometheus renderer for it (the only existing governor CLI surface was
governor list/pause/resume/status). This PR adds the renderer as a prerequisite, following the same local-package convention already established byqueue metrics/ledger metrics.capLimitsisn't itself persisted per-repo by governor-state.js (capUsageis a single global scalar row, not indexed by repo), so the cap-usage ratio uses the fleet-wideDEFAULT_AMS_POLICY_SPEC.capLimitsdefault — the same approximationloop-cli.jsitself already makes for any repo without its own.gittensory-ams.ymloverride.Closes #5187
Test plan
npm run typechecknpm run build:miner(added the new file to the package'snode --checklist)npm run selfhost:validate-observability,npm run docs:drift-check,npm run command-reference:check,npm run miner:env-reference:check,npm run test:miner-packtest/unit/alerts-miner-governor-rate-limit-budget-pressure.test.ts) parses the realalerts.ymland pins both rules'expr/for/severity/annotations to the renderer's exported metric-name constants, plus asserts neither rule's runbook instructs a governor decision-logic changerenderGovernorMetrics/runGovernorMetricscover empty state, global + per-repo bucket ratios, an actionClass with no configured policy (skipped), a malformed per-repo key with no colon separator, deterministic sort order, and cap-usage ratios across all three dimensions