Skip to content

feat(mcp): evaluate when a rented loop should escalate to a human - #5806

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
luciferlive112116:feat/loop-escalation-evaluator
Jul 14, 2026
Merged

feat(mcp): evaluate when a rented loop should escalate to a human#5806
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
luciferlive112116:feat/loop-escalation-evaluator

Conversation

@luciferlive112116

Copy link
Copy Markdown
Contributor

Summary

Closes #4806

Implements the support/escalation-path decision logic: given an already-computed loop outcome, health tier, and operator/customer signals, decide whether a loop needs a human and what action to take — the deterministic core that routes "something's wrong" to a stop-and-review state. Composes with the loop-health evaluator (#4808) on the Rent-a-Loop path #4778.

  • new packages/loopover-engine/src/loop-escalation.ts (pure): evaluateEscalation(input) returns shouldEscalate + action (none/notify/human_review/stop) + severity + reasons, by precedence — a requested stop wins; an errored run or critical health needs a human now; a give-up or customer flag needs a human soon; a soft degradation only notifies. No IO: it decides, the caller wires the action (a stop maps to Incident runbook for the kill-switch #4809's kill-switch once that lands).
  • new loopover_evaluate_escalation MCP tool (src/mcp/server.ts); src/loop-escalation.ts is a thin re-export shim over the engine module.

Scope

  • Conventional Commit title.
  • Focused: the escalation evaluator + its tool, on the merged loop-lifecycle chain.
  • Follows CONTRIBUTING.md; no site/CNAME/VitePress.
  • Linked open issue (Closes Support & escalation path #4806, above).

Validation

  • git diff --check
  • npm run typecheck clean; engine build + build:mcp clean
  • test:coverage on the changed code: loop-escalation.ts 100% lines & branches (24/24); the new src/mcp/server.ts handler + schema + registration fully covered (all 36 changed lines, both branch sides — diff-verified).
  • MCP tool-invariant suites pass with the new tool (mcp-output-schemas "outputSchema on EVERY tool" + "schema-valid structured content").
  • Tests cover every trigger and precedence tier plus the no-escalation path, engine-level and end-to-end through the MCP tool.

If any required check was skipped, explain why:

  • Full test:ci not run end-to-end locally (Linux-only shell/self-host steps on Windows); the change-relevant gates above were validated directly.

Safety

  • No secrets, wallets, hotkeys, trust scores, rewards, private rankings, or private maintainer evidence — the decision is public-safe loop status only, source-free.
  • No auth/cookie/CORS/GitHub App/session change (pure function over caller-supplied data).
  • MCP behavior added + tested (output schema + schema-valid content).
  • No UI changes; no changelog edit.

Notes

Closes JSONbored#4806

Implements the support/escalation-path decision logic: given an already-computed loop outcome, health
tier, and operator/customer signals, decide whether a loop needs a human and what action to take -- the
deterministic core that routes "something's wrong" to a stop-and-review state. Composes with the
loop-health evaluator (JSONbored#4808) on the Rent-a-Loop path JSONbored#4778.

- new packages/loopover-engine/src/loop-escalation.ts (pure): evaluateEscalation(input) returns
  shouldEscalate + action (none/notify/human_review/stop) + severity + reasons, by precedence -- a
  requested stop wins; an errored run or critical health needs a human now; a give-up or customer flag
  needs a human soon; a soft degradation only notifies. No IO: it decides, the caller wires the action
  (a stop maps to JSONbored#4809's kill-switch once that lands).
- new loopover_evaluate_escalation MCP tool (src/mcp/server.ts); src/loop-escalation.ts is a thin
  re-export shim over the engine module.
- tests cover every trigger and precedence tier plus the no-escalation path, at the engine level and
  end-to-end through the MCP tool.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.11%. Comparing base (db74aa3) to head (b0e2c81).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5806   +/-   ##
=======================================
  Coverage   95.11%   95.11%           
=======================================
  Files         586      587    +1     
  Lines       46480    46509   +29     
  Branches    14860    14870   +10     
=======================================
+ Hits        44209    44238   +29     
  Misses       1515     1515           
  Partials      756      756           
Flag Coverage Δ
shard-1 43.76% <86.20%> (+0.02%) ⬆️
shard-2 36.18% <55.17%> (-0.06%) ⬇️
shard-3 32.12% <10.34%> (+0.04%) ⬆️
shard-4 32.81% <10.34%> (-0.02%) ⬇️
shard-5 31.62% <10.34%> (-0.02%) ⬇️
shard-6 44.51% <10.34%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
packages/loopover-engine/src/loop-escalation.ts 100.00% <100.00%> (ø)
src/mcp/server.ts 95.94% <100.00%> (+0.03%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 14, 2026
@loopover-orb

loopover-orb Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-14 19:11:11 UTC

6 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This adds a small, pure escalation-decision function (`evaluateEscalation`) plus wiring through the engine barrel, a re-export shim, and a new MCP tool, mirroring the existing quota/loop-health evaluator pattern in this codebase. The precedence logic (kill > error/critical > abandoned/customer-flagged > degraded > none) is straightforward and correctly implemented as an if/else-if chain, and the accompanying tests exercise every branch and precedence tier at both the engine and MCP-tool level. The `severity`/`action` output schema fields are all `.optional()` in Zod despite always being populated by `evalEscalation`, which is a minor schema-looseness nit rather than a defect.

Nits — 5 non-blocking
  • src/mcp/server.ts's `evaluateEscalationOutputSchema` marks `shouldEscalate`, `action`, `severity`, and `reasons` all `.optional()`, even though `evaluateEscalation` always returns every field — tightening this to required would give MCP clients a stronger contract, consistent with how confidently the tool always returns a complete decision.
  • The external brief's 'deep nesting depth 5' flag on loop-escalation.ts:56 is just a linear if/else-if chain for 5-way precedence, not a real nesting concern — not actionable.
  • `packages/loopover-engine/src/index.ts` and `src/mcp/server.ts` being flagged as long files is pre-existing accumulation, not something introduced by this diff's ~9 and ~36 line additions respectively.
  • Consider making the four fields in `evaluateEscalationOutputSchema` (src/mcp/server.ts) required rather than optional, since `evalEscalation` never omits any of them.
  • If a future PR wires the actual `stop` action to Incident runbook for the kill-switch #4809's kill-switch, make sure the MCP tool's 'it decides, caller wires' contract stays documented at the call site so the separation of concerns doesn't erode.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #4806
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 95 registered-repo PR(s), 42 merged, 39 issue(s).
Contributor context ✅ Confirmed Gittensor contributor luciferlive112116; Gittensor profile; 95 PR(s), 39 issue(s).
Gate result ✅ Passing No configured blocker found.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence. LLM value judgment: moderate — The PR delivers a small, well-tested, pure decision function plus its MCP tool that closes the linked issue #4806 and slots cleanly into the existing loop-lifecycle evaluator pattern (quota/loop-health) without unrelated scope creep.
Linked issue satisfaction

Partially addressed
The PR builds solid deterministic decision logic for when/how a loop should escalate (a real building block for the escalation path) but it is explicitly blocked on #4809's kill-switch and does not implement or simulate an actual end-to-end escalation reaching a human, nor demonstrate the acceptance criterion of reaching a human-reviewable state within a defined target time.

Review context
  • Author: luciferlive112116
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 95 PR(s), 39 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 90f477e into JSONbored:main Jul 14, 2026
16 checks passed
This was referenced Jul 14, 2026
This was referenced Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support & escalation path

1 participant