Skip to content

skill_discovered event re-emitted on every step, inflating telemetry counts #68

@byapparov

Description

@byapparov

Problem

The skill_discovered NDJSON event is emitted once per skill every time the skill tool schema is registered — which happens once per step. This inflates downstream telemetry counts by a factor equal to the number of steps in the session.

Evidence

A recent execution bundled 2 skills into the workspace and ran 19 steps. Telemetry recorded:

skill_discovered: 38
step_start: 19

38 = 2 skills × 19 steps. Exact match.

Root cause

src/tool/skill.ts (SkillTool):

export const SkillTool = Tool.define("skill", async (ctx) => {
  const skills = await Skill.all()

  // Emit per-skill discovery events when descriptions are registered in tool schema
  if (ctx?.sessionID) {
    for (const skill of skills) {
      Bus.publish(Session.Event.SkillDiscovered, {
        sessionID: ctx.sessionID,
        name: skill.name,
        description: skill.description,
        location: skill.location,
      })
    }
  }
  ...
})

The factory runs on every tool-schema registration, so each step re-publishes the full skill list.

Impact

  • Downstream consumers (Session Control executor telemetry, dashboards) misreport "skills used" as a huge number — e.g. a 2-skill bundle appears as 38 "discoveries".
  • Makes it hard to distinguish genuine skill-loading anomalies from normal multi-step runs.
  • The event name suggests one-time discovery, which is what consumers assume.

Suggested fix

Emit skill_discovered once per (sessionID, skillName) — dedupe on the CLI side, either:

  1. Move the emission out of the tool factory into session start / skill-registry initialisation, OR
  2. Track emitted (sessionID, name) pairs in a Set and skip duplicates, OR
  3. Rename the event to skill_available (per-step) and add a new one-shot skill_loaded at session start — but this is a breaking schema change.

Option 1 is cleanest and matches the name's intent.

Version

Observed on @aictrl/cli@0.3.3. Same code path exists in 0.3.2.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions