Skip to content

[Proposal] Optional tools/jev/ adapter — typed-decision pre-filter for triage, PR-management, and security skills #1370

Closed

Description

@onlyarnav

Summary

Add an optional, adopter-opt-in tool adapter (tools/jev/) around TypeSafe's
Jev API (api.typesafe.ai/v1/systemone) so skills can get a fast, cheap,
typed yes/no/choice/score decision instead of a full agent reasoning pass,
for the narrow, bounded classification and gating steps that already exist
in several skills. This is a new external-system adapter alongside
tools/github/, tools/gmail/, etc. — not a replacement for any agent
harness.

Motivation

Several skills already do the kind of bounded classification Jev is built
for — and PR triage/review is one of the four core streams Magpie's
MISSION.md names, so this is squarely in scope rather than a side interest:

  • pr-management-triage / issue-triage sorting the queue
  • security-issue-triage classifying inbound reports
  • The "narrowly-scoped fix-and-merge automation" mode on the roadmap needs
    a safe go/no-go gate

Today these run as full LLM/agent reasoning calls. For the bounded part (a
closed label set, a risk score, a yes/no gate) Jev returns a calibrated
probability in ~70-500ms at a fraction of the token cost, with typed output
that removes fragile JSON-parsing from the skill's response handling.

Proposed shape

  • New tools/jev/tool.md + a thin, stateless HTTP adapter — same pattern
    as tools/github/. Declares its own adopter-side variable (JEV_API_KEY);
    unset = tool unavailable, skill falls back to its current agent-only path
    unchanged.
  • First candidate call sites (opt-in per skill, not wired by default):
    • pr-management-triage — a choice call to pre-sort a PR into existing
      triage buckets alongside the agent pass
    • A score/noul gate ahead of any future auto-merge step — escalates to
      human review below a confidence threshold, never used to loosen the
      human-in-the-loop requirement
    • security-issue-triage — a noul call as a first-pass
      "is this security-sensitive" filter

Implementation plan

  • PR 1: tools/typed-decision/ — generic contract + Jev provider
  • PR 2: opt-in choice pre-filter in pr-management-triage
  • PR 3: eval script + results write-up

How this respects RFC-AI-0004

  • HITL — Jev never mutates state; it only informs whether a skill
    proceeds or escalates. Proposal/confirm/apply is untouched.
  • Vendor neutrality — a second, fully optional vendor (a decision
    model, not an agent harness); every skill must behave identically with
    the adapter absent.
  • Privacy by design — Jev calls see the same content an agent call
    would, so they route through the existing tools/privacy-llm/ gate:
    only content a project's PMC has already approved for a non-exempt LLM
    reaches it.
  • Conversational/correctable — threshold and enable/disable per skill
    go through the existing .apache-magpie-overrides/ mechanism.

Open questions

  1. Does a paid, closed-weights, early-access API (launched Sept 14, 2026;
    pricing/limits explicitly "subject to change") belong in the framework's
    tool set at all, even opt-in?
  2. Land as one general tools/jev/ adapter usable by any skill, or scoped
    to a single skill first (e.g. just pr-management-triage) to prove the
    pattern before wiring it elsewhere?
  3. Does this need an RFC (a new normative allowance for non-agent model
    vendors), or is a tool adapter + per-skill PRs enough?

Non-goals

  • Not a replacement for any harness in the Agent harnesses table.
  • No skill should ever require Jev to be configured.

Activity

  1. potiuk commented on Sep 24, 2026

    @potiuk
    Member

    Oh absolutely. Jev is all the hot topic now. Go for it @onlyarnav

  2. added 9 commits that reference this issue on Sep 26, 2026
    95e7e1e
    9fb5488
    5a06526
    6b5634d
    1e22250
    bd25597
    c358f33
    e031310
    859afa8
  3. added 5 commits that reference this issue on Oct 4, 2026
    f90beb0
    747f768
    713750c
    6221594
    b9b2ce2
  4. 5 remaining items

  5. removed
    family:docsDocs, MISSION.md, READMEs
    family:ci.github workflows, prek, validators
    substrate:framework-devTool substrate: build / validate / eval the framework itself
    on Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions