feat(tools): per-tool descriptor and skill-use tracking for every tool (#1221) by SaulMoro · Pull Request #1245 · Tencent/teamai-cli

SaulMoro · 2026-10-20T20:05:54Z

Summary

Step A1 of #1221: one typed descriptor per tool, and skill usage recorded the same way for every tool.

flowchart LR
    subgraph callers ["callers"]
        hooks["hook-dispatch<br/>post-tool-use '*', prompt-submit"]:::added
        init["init<br/>home detection"]:::added
        pull["pull, uninstall, doctor,<br/>models switch, local agent"]
        test["matrix test"]:::todo
    end
    subgraph core ["src/tools.ts"]
        desc["TOOLS<br/>home, skillLoad"]:::added
        rest["channels, support"]:::todo
        locate["locate(tool, resource, install, host)"]:::todo
    end
    use["skill-use<br/>skill tool, SKILL.md read, mention<br/>installed only, once per turn"]:::added
    host{{"Host"}}:::todo
    page["docs/guide/tool-support.md"]:::todo
    old["per-tool Skill hook,<br/>SKILL_TOOL_MATCHER"]:::removed
    res["per-tool resolvers"]
    hooks --> use --> desc
    init --> desc
    pull & test -.-> locate
    locate -.-> desc
    locate -.-> rest & host
    rest -.-> page
    pull --> res
    hooks -.-> old
    classDef added fill:#2f8f4f,stroke:#1f6b39,color:#fff;
    classDef todo fill:#5c6670,stroke:#3d454c,color:#fff,stroke-dasharray:4 3;
    classDef removed fill:#a33a3a,stroke:#7a2a2a,color:#fff,stroke-dasharray:4 3;

Green: this PR. Gray: later steps of #1221 (A2 Host and locate, then B1-B4, C, E, D). Red: removed.

A skill counts when the agent calls its skill tool, reads an installed SKILL.md (shell reads included), or the prompt names it: /name, Codex $name, Pi /skill:name. Once per turn, installed skills only.
Fixes Copilot, Pi, OMP, dsh, OpenCode 2.x and Codex, which record no skill usage today.
OpenCode's plugin sends raw tool names; Skill is still accepted for one release.
A team that disabled Hook dispatch post-tool-use Skill keeps tracking off.

Type of Change

Bug fix (non-breaking change that fixes an issue)
New feature (non-breaking change that adds functionality)
Breaking change (fix or feature causing existing behavior to change)
Documentation only
Refactor / internal cleanup

Evidence

Before: on main, the Codex, Pi, Copilot, OMP, dsh and OpenCode 2.x payloads captured for #1221 record no skill use.
After: the built CLI at 90c1b356, isolated HOME, user scope, teamai hook-dispatch ... --stdin:

codex   "$hello-skill ..." + cat SKILL.md x2, same turn   -> 1 (codex)
codex   cat ~/.codex/skills/hello-skill/SKILL.md, new turn -> 1 (codex)
claude  Skill call                                       -> 1 (claude)
claude  Skill call, skill not installed                  -> 0

Earlier payloads from real runs (Pi /skill:x, stale --matcher Skill hook, invalid team hook or builtin block, retired key, hooks.yaml syntax error) are covered by skill-use.test.ts and hooks-skill-retire.test.ts.

Test Plan

npx tsc --noEmit passes
npm run lint passes
npx vitest run passes (11459 passed, 21 skipped)
Added/updated tests for the change: skill-use.test.ts, hooks-skill-retire.test.ts, tools.test.ts
npm run build, then the full npm run test:e2e: 1394 passed, 27 skipped, 2 failed. The 2 are the typed-terminal Ctrl-C cases in workflow-signals.test.ts; they fail the same way on main (c6bd1264) on this machine, and pass in main's CI.
Not run here: real Copilot, CodeBuddy, Muse, dsh and Pi hosts, Windows, the gitlab and github providers.

Merge Danger

Door: two-way. Reverting restores the Skill hook on the next pull.

Blast Radius: usage counts.

Counts change on purpose: once per turn, installed skills only, /name only at the start.
Codex hooks.json trust keys are positional, so removing the Skill entry re-trusts the later entries on pull.
Known gaps: hosts with no turn id (Pi, OMP, OpenCode) can lose a credit when a detached hook lands after the next prompt; Cursor may count once per conversation.

Related Issues

Part of #1221.

github-actions · 2026-10-20T20:38:09Z

[P1 blocking] Prompt deduplication drops skill-use webhooks — src/hook-handlers.ts:374: A /tdd prompt credits the installed skill without sending a webhook. The subsequent Skill call in the same turn returns no new credit, so this loop sends nothing either. Existing webhook users therefore lose notifications for slash-invoked skills. Send the webhook when the prompt wins the credit, or deduplicate webhook delivery separately.
[P2 non-blocking] Quoted HOME paths are not tracked — src/utils/shell-command.ts:131: Expansion runs only outside quotes. A successful cat "$HOME/.codex/skills/hello-skill/SKILL.md" leaves $HOME unresolved and records no skill use, unlike the equivalent unquoted command. Handle $HOME and ${HOME} inside double quotes while preserving single-quoted literals.
[P2 non-blocking] Name-only mentions accept another tool’s installations — src/skill-use.ts:131: The existence gate searches every tool’s roots without receiving the calling tool. Consequently, Codex credits $pi-only when that skill exists solely under ~/.pi/agent/skills, contradicting the documented “installed for the tool” restriction. Restrict name-only candidates to the caller’s discoverable roots.

The PR description supplies sufficient built-CLI/e2e evidence. No testing-record finding; nothing from the PR was run, built, or installed.

github-actions · 2026-10-20T22:17:17Z

No new findings in the scoped diff.

Earlier findings are resolved:

Prompt deduplication now claims webhook delivery separately from usage credit.
$HOME and ${HOME} expand inside double quotes while single-quoted literals remain unchanged.
Name-only candidates now use the calling tool’s skill roots.

The PR description documents sufficient built-CLI/e2e verification. No testing-record finding; deferred provider/agent coverage is not blocking.

Nothing from the PR was run, built, or installed.

…nce (Tencent#1221) TOOLS records each built-in tool's home directory and how its agent loads a skill (dedicated skill tool, prompt mention, or 'unknown'), with an evidence class per value. ToolId is derived from DEFAULT_TOOL_PATHS, so a new built-in tool fails tsc until it has a descriptor. KNOWN_AGENTS user skills paths for pi, omp and opencode now match their user-scope defaults; home detection and the init picker read home.dir so Pi detection does not move. Tests pin the tool lists to ToolId, every user-scope path under its home, and skillLoad to the tools that receive the wildcard post-tool-use hook.

Tencent#1221) One background skill-use handler on the wildcard post-tool-use pass credits a skill when the call is the tool's skill tool (TOOLS[tool].skillLoad) or a read of an installed skill's SKILL.md (classifyToolCall, shell reads with ~/ and $HOME/ included). Prompt-submit credits /name at the start, Codex $name, and the block Pi expands /skill:name into. Each skill counts once per session turn and only when installed under a root skill delivery uses. Removes the per-tool Skill hook, SKILL_TOOL_MATCHER, and the OpenCode matcher passes; OpenCode sends raw tool names, with a one-release Skill alias. A team that disabled the retired built-in key keeps tool-call tracking off.

github-actions · 2026-10-21T03:20:19Z

No new findings in the scoped diff.

Earlier findings are resolved:

Webhook delivery is claimed separately from usage credit.
$HOME and ${HOME} expand inside double quotes; single-quoted literals remain unchanged.
Name-only candidates use the calling tool’s skill roots.

The PR description documents sufficient built-CLI/e2e verification. Deferred provider/agent coverage is not blocking.

Nothing from the PR was run, built, or installed.

…tor (#1221) The env, workflows, recall, stats, AI backend, votes judge and HTTP agent hook rows of #1221's matrix become a required `features` facet, one `SupportCell` per row: the mark and the evidence letter, as the matrix writes them. A cell a later fix changed (#1193, #1224, #1245, #1374) says so in its note and keeps teamai's-code-only evidence until it is rerun. Data only: nothing reads it but the support page.

…Code's inlined rules (#1221) Usage: skill use is recorded for OpenCode, Pi, Oh My Pi and DeepSeek Harness since #1245 (`features.stats`). Sessions: `teamai session` reads OpenCode (`TOOLS.opencode.sessions`). Rules: JoyCode's user rules are inlined into rules.txt, so they stay on, as for every other tool marked ✓¹.