Skip to content

feat(commands): intent-classification router for unrecognized @gittensory mentions #4596

Description

@JSONbored

Part of #4594 (Option 3 — near-zero-risk fast-follow to #4595).

Context

Today, an @gittensory mention whose verb token doesn't resolve to a known command is silently
downgraded to { name: "help", unknownVerb: rawVerbToken } (src/github/commands.ts:278), which
triggers a fuzzy "did-you-mean" hint (buildDidYouMeanSections, commands.ts:766). This closes the
literal "contributor doesn't know the exact command syntax" gap without the larger safety surface
of open-ended generation (#4594's rejected Option 2): the classifier only ever picks a label from a
closed, pre-enumerated set — it never generates a new factual claim, so there is no hallucination surface
to defend against, only mis-routing (annoying, not misleading or unsafe).

Requirements

  1. Trigger: same detection surface as the existing unrecognized-verb fallback — an @gittensory
    mention with an unrecognized verb (or no verb) plus non-trivial trailing free text, when the new
    capability flag is enabled.
  2. Closed-set classifier, not a generator: maps free text to the single closest match among the
    existing Q&A commands only (preflight, blockers, duplicate-check, next-action,
    reviewability, repo-fit, packet, ask, and — once feat(commands): add @gittensory chat <question> — grounded LLM Q&A via local Ollama #4595 ships — chat) or returns "no match."
    On a match, dispatch through the exact same buildMentionCommandBundle path those commands already
    use (processors.ts:14696-14735) — the answer content is 100% pre-existing deterministic output, only
    reachable without knowing the exact verb. On no-match, fall back to the existing
    buildDidYouMeanSections hint unchanged.
  3. Hard runtime allowlist, not prompt-instructed containment: the classifier's output type is a
    literal union of the Q&A catalog's command names — never action-catalog names. Any output outside
    that allowlist (including a prompt-injection attempt to make the classifier "choose"
    review/pause/gate-override/etc.) must be rejected by a hard runtime check before dispatch, not
    relied on the prompt to refuse. This must be exhaustively testable: feed the classifier adversarial
    inputs and assert its raw output is filtered to the closed Q&A-catalog-name set in 100% of cases —
    this is a much smaller, fully enumerable test surface than filtering free-form prose.
  4. New capability flag: advisoryAiRouting.intentRouting: boolean (default false) — same shape as
    feat(commands): add @gittensory chat <question> — grounded LLM Q&A via local Ollama #4595's chatQa flag (type + normalizer + engine-package twin + focus-manifest merge). Same
    no-frontier-fallback divergence as feat(commands): add @gittensory chat <question> — grounded LLM Q&A via local Ollama #4595: never falls back to the frontier chain; decline/no-op
    (silently fall through to the existing did-you-mean hint) if the flag is off or env.AI_ADVISORY is
    unconfigured.
  5. Rate-limit tier: the classifier call is cheap (short prompt, tiny closed-set output) — still add it
    to AI_COST_BEARING_COMMANDS (commands.ts:321-330) anyway for budget-ledger consistency, since it
    still calls the AI binding, even though its cost profile is much smaller than feat(commands): add @gittensory chat <question> — grounded LLM Q&A via local Ollama #4595's generation call.
  6. Show the interpretation: post the matched command name alongside the answer (e.g. "interpreted your
    question as @gittensory blockers") so a wrong match is immediately visible/correctable. Keep a
    "no confident match" threshold that prefers the existing did-you-mean fallback over a low-confidence
    guess.

Deliverables

  • Small classifier function (short prompt + closed JSON output schema)
  • Hard allowlist-enforcement check (output filtered to Q&A-catalog names only, tested exhaustively)
  • intentRouting capability flag: type + normalizer + engine-package twin + focus-manifest merge
  • Wiring into the existing unrecognized-verb branch of the parser/dispatcher
    (buildDidYouMeanSections's call site, commands.ts:766)
  • AI_COST_BEARING_COMMANDS membership
  • "Interpreted as" prefix on the routed answer card
  • Unit tests: classifier allowlist enforcement (adversarial-input corpus), routing accuracy on a
    sample question set, fallback-to-did-you-mean on no-match, 99% patch coverage per house rules
  • OpenAPI schema entry + config/examples/gittensory.full.yml documentation

Expected outcome

A contributor who @gittensorys a question using natural language instead of the exact command syntax
(e.g. "why is this stuck" instead of @gittensory blockers) gets routed to the correct existing answer
automatically, with the interpretation shown so a wrong guess is obviously correctable — zero new
generated content, zero new injection surface, entirely reusing existing, already-tested answer
rendering. Config-driven so any self-hoster can enable/scope it independently of #4595.

Links + resources

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

Status
In Progress

Relationships

None yet

Development

No branches or pull requests

Issue actions