Skip to content

feat(ams): page PagerDuty on miner kill-switch trips - #7701

Closed
jsdevninja wants to merge 1 commit into
JSONbored:mainfrom
jsdevninja:feat/ams-kill-switch-pagerduty-v3
Closed

feat(ams): page PagerDuty on miner kill-switch trips#7701
jsdevninja wants to merge 1 commit into
JSONbored:mainfrom
jsdevninja:feat/ams-kill-switch-pagerduty-v3

Conversation

@jsdevninja

Copy link
Copy Markdown
Contributor

Summary

  • On AMS kill-switch trip, miner recordMinerKillSwitchTransition fires PagerDuty Events API v2 (same LOOPOVER_ENABLE_PAGERDUTY / PAGERDUTY_ROUTING_KEY / enqueue / dedup_key contract as ORB notify-pagerduty). Resumes stay silent; paging is fire-and-forget so it cannot block the ledger write.
  • Pure trip detection (buildMinerKillSwitchPagerDutyAlert) lives at the miner IO seam (not a hosted dead export). Runbook Detection documents the automated page.
  • Replaces closed feat(ams): page PagerDuty on miner kill-switch trips #7685 / feat(ams): page PagerDuty on miner kill-switch trips #7699 (hosted unused wrapper; then MCP build-drift cascade when engine changed).

Closes #7666

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (Closes #7666).

Validation

  • git diff --check
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

  • Focused: miner-governor-kill-switch + kill-switch-incident-runbook (25 tests green). Full test:ci left to CI.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
  • API/OpenAPI/MCP behavior is updated and tested where needed.
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
  • Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

UI Evidence

N/A — docs runbook text + miner alerting only; no visible UI chrome.

Notes

  • Analogue: src/services/notify-pagerduty.ts Events API v2 contract.
  • No hosted notifyMinerKillSwitchPagerDuty wrapper: AMS trips only run in the miner process.

On ledger trip, fire Events API v2 using the same LOOPOVER_ENABLE_PAGERDUTY / PAGERDUTY_ROUTING_KEY contract as notify-pagerduty. Resumes stay silent; paging is fire-and-forget. Pure alert builder lives at the miner IO seam so engine rebuilds do not trip MCP emit drift.

Co-authored-by: Cursor <cursoragent@cursor.com>
@jsdevninja
jsdevninja requested a review from JSONbored as a code owner July 21, 2026 09:43
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 21, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
6635 1 6634 7
View the top 1 failed test(s) by shortest run time
test/unit/check-build-drift-script.test.ts > check-build-drift script > CLI (real subprocess) > reports the real repo's loopover-mcp output as clean
Stack Traces | 0.0877s run time
Error: Command failed: .../hostedtoolcache/node/22.23.1.../x64/bin/node scripts/check-build-drift.mjs mcp
check-build-drift: packages/loopover-mcp's committed .js/.d.ts is stale relative to its .ts source -- run `npm run build:mcp` and commit the result:
M .../loopover-mcp/bin/loopover-mcp.js

 ❯ test/unit/check-build-drift-script.test.ts:105:22

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { status: 1, signal: null, output: [ null, '', 'check-build-drift: packages/loopover-mcp\'s committed .js/.d.ts is stale relative to its .ts source -- run `npm run build:mcp` and commit the result:\nM .../loopover-mcp/bin/loopover-mcp.js\n' ], pid: 6423, stdout: '', stderr: 'check-build-drift: packages/loopover-mcp\'s committed .js/.d.ts is stale relative to its .ts source -- run `npm run build:mcp` and commit the result:\nM .../loopover-mcp/bin/loopover-mcp.js\n' }

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 21, 2026
@loopover-orb

loopover-orb Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Caution

🛑 LoopOver review result - fixes required

Review updated: 2026-07-21 09:57:03 UTC

7 files · 1 AI reviewer · no blockers · CI failing · blocked

🛑 Suggested Action - Fix Blockers

Review summary
This adds a fire-and-forget PagerDuty page on AMS miner kill-switch TRIP (not resume), mirroring the existing hosted `notify-pagerduty.ts` Events API v2 contract (flag/routing-key/enqueue/dedup_key) but without the hosted module's cooldown/min-severity/per-repo-key machinery, which the PR explicitly documents as a deliberate simplification since the miner has no Worker Env/D1. The pure `buildMinerKillSwitchPagerDutyAlert` correctly gates on trip-into-active-scope only (null on resume and no-op transitions), and `recordMinerKillSwitchTransition` appends the ledger write first, then fires `notify` wrapped in both a sync try/catch and a `.catch` on the resulting promise so a paging failure can never block or throw past the ledger write. Test coverage is thorough — sync/async throws, non-Error throws, non-ok status, default env/notify wiring, and all branch permutations of the alert builder are exercised against the real code paths, not fabricated ones.

Nits — 5 non-blocking
  • packages/loopover-miner/lib/governor-kill-switch.ts:94 hardcodes the PagerDuty Events API host, matching src/services/notify-pagerduty.ts's existing pattern but duplicating the URL/magic numbers (1024, 5000, 200) in a second file rather than sharing a constant.
  • The miner-side notify only supports a single global PAGERDUTY_ROUTING_KEY, unlike the hosted module's per-repo PAGERDUTY_REPO_ROUTING_KEYS map — intentional per the docs comment, but worth a runbook note for operators expecting per-repo routing.
  • governor-kill-switch.js is a checked-in compiled artifact mirroring the .ts source; confirm your build step is what regenerates it and it isn't hand-edited, since a future .ts change could silently drift from it.
  • The `validate` and `validate-tests (3)` checks failed with no detail provided in the CI status, so the cause could not be verified from what's given here.
  • Consider factoring the shared Events API v2 POST body construction (routing_key/event_action/payload shape) into one helper used by both notify-pagerduty.ts and governor-kill-switch.ts to avoid the two implementations drifting apart over time.

CI checks failing

  • validate
  • validate-tests (3)

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #7685, #7666
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (2 linked issues).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 213 registered-repo PR(s), 136 merged, 37 issue(s).
Contributor context ✅ Confirmed Gittensor contributor jsdevninja; Gittensor profile; 213 PR(s), 37 issue(s).
Improvement ✅ Minor risk: clean · value: minor · LLM: moderate
Linked issue satisfaction

Addressed
The PR adds a pure buildMinerKillSwitchPagerDutyAlert function, wires recordMinerKillSwitchTransition to page via notifyMinerKillSwitchPagerDuty on trip (using the same LOOPOVER_ENABLE_PAGERDUTY/PAGERDUTY_ROUTING_KEY contract as ORB) while staying silent on resume, and documents the automated page in the runbook, closely matching all three bullet points in the issue's summary.

Review context
  • Author: jsdevninja
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: JavaScript, Swift, C, CSS, MDX, Python, TypeScript, Vue
  • Official Gittensor activity: 213 PR(s), 37 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

Visual preview
Route Viewport Before (production) After (this PR's preview) Diff
/ desktop before /
before /
after /
after /
/ mobile before / (mobile)
before / (mobile)
after / (mobile)
after / (mobile)

Click any thumbnail to open the full-size screenshot. Before = production · After = this PR's preview deploy.

Scroll preview
Route Before (production) After (this PR's preview)
/ before / (scroll)
before / (scroll)
after / (scroll)
after / (scroll)

A short scroll-through clip (desktop) — click either thumbnail to open the full animation. Evidence for scroll-linked behavior a single screenshot can't show.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb

loopover-orb Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

LoopOver is closing this pull request on the maintainer's behalf (CI is failing (validate, validate-tests (3))). This is an automated maintenance action — to pursue this change, please open a new pull request with the issues resolved. Closed PRs may be analyzed later to improve review accuracy, but they are not automatically reopened or re-reviewed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wire real PagerDuty alerting into AMS kill-switch trips (currently ledger-only, no paging)

1 participant