Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 65 additions & 4 deletions docs/reference/cost-ledger-module.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@
`hooks/mod/cost-ledger.ts` is mh's one hooks module (a Claude Code mod, GH #323). It is an
observer: it never denies, asks, rewrites or answers for core. The shell hooks stay the base:
`cost-tracker.sh`, `costs.jsonl` and every gate are unchanged, and the transcript-priced total in
`costs.jsonl` stays the authoritative session cost (GH #334, option A). The module adds one
thing: a cross-check against Claude Code's own cost ledger.
`costs.jsonl` stays the authoritative session cost (GH #334, option A). The module adds three
things: a cross-check against Claude Code's own cost ledger, a per-load marker row (#444), and
in an mh checkout a cached-vs-repo version status and stale-skill toast (#442).

## What it records

Expand Down Expand Up @@ -36,6 +37,59 @@ Names verified against the `claude-code.d.ts` that CLI 2.1.288 writes: `turn.com
streaming event per model request, not per turn. `session.measure` also carries `cost` and fires
after each main-thread turn, but has no turn id and was not probed under `claude -p`.

## Load marker (GH #444)

The first hook call of each module load (`session.start`, `skill.prompt` or `turn.complete`,
whichever runs first) appends one row to the same `cost-ledger.jsonl`:

```
{ t, session_id, loaded: true, cli, mh }
```

`cli` is `$.session.version().version`; `mh` is the version in `$.plugin.root`'s
`.claude-plugin/plugin.json`, the copy that actually runs. A field that cannot be read is left
out; a failed append is a `$.ui.log` line. A reload writes a second marker for the same session.
It goes to the ledger file, not `$.ui.log`, because under `claude -p` the UI log only reaches the
debug log. With modules off nothing is written, and that absence is the signal: "the kill switch
served off" reads apart from "loaded but idle" (a marker and no `usd` rows).

## Cached-vs-repo status (GH #442)

Only when the session's project root (`$.session.root()`) holds a `.claude-plugin/plugin.json`
whose `name` matches `$.plugin.name`, i.e. an mh checkout or worktree. Anywhere else it does
nothing.

- **`session.start`:** one `$.ui.status` line comparing the cached version (`$.plugin.root`) with
the repo manifest's: `mh 1.1.174: cache = repo version (...)` or `mh: cache 1.1.173 is stale,
repo 1.1.174 (...)`. Equal versions do not prove equal content (same-version edits are no-ops),
which is what the skill check is for.
- **`skill.prompt`:** finds the skill's `SKILL.md` through the cached manifest's `skills` entries
(`<entry>/<name>/SKILL.md`, or `<entry>/SKILL.md` when the entry is the skill's own folder; a
`plugin:` prefix on the name is dropped) and compares it byte for byte with the same path in
the repo. A difference is one `$.ui.toast` per skill per module load saying the cached copy runs.
A skill not in mh, or missing from the repo, gets nothing. The prompt text passes through
untouched.

This is the in-session answer to "which mh copy runs" (`docs/reference/repo-gotchas.md`); a
`Skill()` call alone runs the cached copy and cannot confirm a repo edit.

Names verified against the `claude-code.d.ts` that CLI 2.1.289 writes, and listed by
`claude plugin validate .` 2.1.289 as the module's calls: `session.start` (`SessionStartInput`),
`skill.prompt` (`SkillPromptInput`: `skill`, `text`; result `{ text }`), `$.plugin.root` /
`$.plugin.name`, `$.session.root()`, `$.session.version()` (`SessionVersion.version`),
`$.fs.read(path)`, `$.ui.status(text)`, `$.ui.toast(text, { timeoutMs })`.

**Live probe: not run.** A dispatched subagent cannot start `claude -p` (mh's
`gate:bash:irrecoverable` blocks nested sessions), so three things rest on the d.ts alone: that
`$.plugin.root` is the folder above `.claude-plugin/` (the d.ts example
`${$.plugin.root}/hooks/weights.bin` implies it; if wrong, every read misses and the module shows
nothing, marker `mh` field absent), whether `skill.prompt`'s `skill` carries the `mh:` prefix
(both forms are handled), and that `session.start` fires under `-p`. To probe from a main
session: copy the repo to a scratch folder, set its `plugin.json` version to `0.0.1` and edit one
`SKILL.md`, run `claude -p "/mh:cost-report" --plugin-dir <copy> --debug-file <log>` from the
repo, then check for a `loaded` row in `cost-ledger.jsonl` with `mh: "0.0.1"` and the status and
toast text in the debug log.

## How cost-report reads it

`skills/meta/cost-report/scripts/cost-report-dedup.js` prints one `ledger cross-check:` line: the
Expand All @@ -45,10 +99,17 @@ and ignores the stored `delta`, which restarts when the module reloads. The rule
three cases are in `skills/meta/cost-report/references/data-model.md`, section
"Gap to Claude Code's own ledger".

It also prints one `module loaded:` line from the load markers: `module loaded: N of M sessions
since the first load marker (<date>); ...`, where M counts the `costs.jsonl` sessions whose last
row is at or after the first marker (older sessions predate it) and N those with a marker. With no
marker at all it prints `module loaded: no load marker (...)`. Marker rows carry no `usd` and are
not counted in the no-finite-usd warning.

## When modules are off

Nothing is written, and the only loss is the cross-check line: cost-report prints
`ledger cross-check: no ledger data (...)`, not zero and not an error. Modules are off on CLIs
Nothing is written, and nothing else changes: no status line, no toast, no marker. cost-report
prints `ledger cross-check: no ledger data (...)`, not zero and not an error, and the session is
missing from the `module loaded:` count. Modules are off on CLIs
before 2.1.287, under `--bare`, `--safe-mode`, `disableAllHooks`, `allowManagedHooksOnly`,
`allowManagedModsOnly`, in an untrusted workspace, and when Anthropic's remote switch serves off.
A reload after such a flip drops the module mid-session.
Expand Down
3 changes: 2 additions & 1 deletion docs/reference/repo-gotchas.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,8 @@ never runs at all and nothing server-side notices. Run the verify command above
Read `mh_version` in this session's last row of `~/.local/share/kbg/metrics/costs.jsonl` that has
one (a Codex row has none), or the version in the hook path Claude Code prints around a gate deny. `~/.claude/metrics/costs.jsonl` is a stale
file with no `mh_version`. A wrong probe gave a wrong answer twice: the env var (2026-09-21) and the
stale file (2026-09-30).
stale file (2026-09-30). In an mh checkout with modules on, the hooks module's status line and
stale-skill toast say it too (`docs/reference/cost-ledger-module.md`, GH #442).
- **A brand-new skill or agent no longer fails checks 02 and 03.** The plugin cache is built from
committed state, so a component added after the commit the cache was built from cannot be in it.
Checks 02 (skill loadability) and 03 (agent loadability) report such a component as INFO when its
Expand Down
117 changes: 105 additions & 12 deletions hooks/mod/cost-ledger.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import type { Register } from 'claude-code'
import type { EngineInterface, Register } from 'claude-code'

// mh's one hooks module (GH #323, #334): an observer that records Claude Code's own cost ledger
// ($.session.usage().cost.usd, what /cost totals) after each main-loop turn, so cost-report can
// cross-check the transcript-priced costs.jsonl against it. Reference:
// docs/reference/cost-ledger-module.md.
// cross-check the transcript-priced costs.jsonl against it. It also writes one load-proof row per
// module load (#444) and, in an mh checkout, shows whether the cached copy matches the repo
// (#442). Reference: docs/reference/cost-ledger-module.md.
//
// Observer only: it never denies, asks, rewrites or answers for core. Every step sits in one
// try/catch and the hook always ends in `next(e)`, so a failure here costs one row, never the turn.
Expand All @@ -28,8 +29,108 @@ export const ledgerDelta = (prev: number | undefined, usd: number) =>
// deltas start to matter.
let prev: number | undefined

const append = async ($: EngineInterface, row: string) => {
try {
const r = await $.process.run(['sh', '-c', APPEND, 'sh', row], { timeoutMs: 2000 })
if (r.exitCode === 0) return true
$.ui.log(`cost-ledger append exited ${r.exitCode}: ${r.stderr.trim().slice(0, 200)}`)
} catch (err) {
$.ui.log(`cost-ledger append failed: ${String(err).slice(0, 200)}`)
}
return false
}

type Manifest = { name?: string; version?: string; skills?: string[] }
const readText = async ($: EngineInterface, path: string): Promise<string | undefined> => {
try {
return await $.fs.read(path)
} catch {
return undefined
}
}
const readManifest = async ($: EngineInterface, root: string): Promise<Manifest | undefined> => {
try {
return JSON.parse((await readText($, `${root}/.claude-plugin/plugin.json`)) ?? '')
} catch {
return undefined
}
}
// The session's project root and its manifest, when it is a checkout of this plugin (an mh repo
// or worktree); undefined anywhere else.
const mhRepo = async ($: EngineInterface) => {
const root = await $.session.root()
const manifest = await readManifest($, root)
return manifest?.name === $.plugin.name ? { root, manifest } : undefined
}

// GH #444: one load-proof row per module load, from whichever hook runs first, so a session with
// modules off (no row) reads apart from one where the module loaded and stayed idle. A reload
// writes another; the report counts sessions, not rows.
let marked = false
const markLoaded = async ($: EngineInterface) => {
if (marked) return
marked = true
try {
const [session, cli, mh] = await Promise.all([
$.session.id(),
$.session.version().then(v => v.version, () => undefined),
readManifest($, $.plugin.root).then(m => m?.version),
])
await append($, JSON.stringify({ t: new Date().toISOString(), session_id: session, loaded: true, cli, mh }))
} catch (err) {
$.ui.log(`cost-ledger load marker failed: ${String(err).slice(0, 200)}`)
}
}

// GH #442: skills whose repo SKILL.md differs from the cached copy that runs, toasted once each.
const toasted = new Set<string>()

export const register: Register = on => {
on('session.start', async ($, e, next) => {
await markLoaded($)
try {
const repo = (await mhRepo($))?.manifest
const cache = repo?.version ? await readManifest($, $.plugin.root) : undefined
if (cache?.version && repo?.version)
$.ui.status(
cache.version === repo.version
? `${$.plugin.name} ${cache.version}: cache = repo version (same-version edits still need a bump)`
: `${$.plugin.name}: cache ${cache.version} is stale, repo ${repo.version} (bump + claude plugin update)`,
)
} catch (err) {
$.ui.log(`cost-ledger version status failed: ${String(err).slice(0, 200)}`)
}
return next(e)
})

on('skill.prompt', async ($, e, next) => {
await markLoaded($)
try {
const name = e.skill.split(':').pop() ?? ''
const root = name && !toasted.has(name) ? (await mhRepo($))?.root : undefined
if (root) {
// plugin.json's `skills` lists parent folders (./skills/meta/) and skill folders alike.
const dirs = ((await readManifest($, $.plugin.root))?.skills ?? []).map(d => d.replace(/^\.\//, '').replace(/\/$/, ''))
const rels = dirs.flatMap(d => [`${d}/${name}/SKILL.md`, ...(d.split('/').pop() === name ? [`${d}/SKILL.md`] : [])])
for (const rel of rels) {
const cached = await readText($, `${$.plugin.root}/${rel}`)
if (cached === undefined) continue
const current = await readText($, `${root}/${rel}`)
if (current !== undefined && current !== cached) {
toasted.add(name)
$.ui.toast(`${name}: the cached copy runs; repo ${rel} differs (bump + claude plugin update)`, { timeoutMs: 8000 })
}
break
}
}
} catch (err) {
$.ui.log(`cost-ledger skill check failed: ${String(err).slice(0, 200)}`)
}
return next(e)
})

on('turn.complete', async ($, e, next) => {
await markLoaded($)
if (e.agentId !== undefined) return next(e)
let session = 'unknown'
let row: string | undefined
Expand All @@ -48,15 +149,7 @@ export const register: Register = on => {
} catch (err) {
row = JSON.stringify({ t: new Date().toISOString(), session_id: session, turn_id: e.turnId, error: String(err).slice(0, 200) })
}
if (row !== undefined) {
try {
const r = await $.process.run(['sh', '-c', APPEND, 'sh', row], { timeoutMs: 2000 })
if (r.exitCode !== 0) $.ui.log(`cost-ledger append exited ${r.exitCode}: ${r.stderr.trim().slice(0, 200)}`)
else if (usd !== undefined) prev = usd
} catch (err) {
$.ui.log(`cost-ledger append failed: ${String(err).slice(0, 200)}`)
}
}
if (row !== undefined && (await append($, row)) && usd !== undefined) prev = usd
return next(e)
})
}
1 change: 1 addition & 0 deletions skills/meta/cost-report/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ Quote the script's numbers as printed. Add only what a reader needs to interpret
The line sets it against Claude Code's own ledger (what `/cost` shows) for sessions present in
both logs; the gap is spend with no transcript line (about 17% on 2026-10-02). `no ledger data`
means mh's hooks module was not running (a CLI before 2.1.287, or mods off), not zero spend.
`module loaded: N of M sessions` is the module's load rate since its first load marker.
- **Handoff cost (2026-09-20+ rows) prices reading a subagent's return.** Median/p90 tokens
main spends per subagent return (`verify_tokens`), plus returns per orchestrator turn. Rows
before this restore carry no `verify_per_return` and the section is omitted if none do.
Expand Down
9 changes: 8 additions & 1 deletion skills/meta/cost-report/references/data-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,14 @@ report then prints one line after `total:`:
- **Data on both sides:** `ledger cross-check: transcript $X vs ledger $Y over N sessions;
unattributed $Z (P% of ledger)`. Only sessions present in both files count; sessions on one
side only are listed as `left out: A transcript-only, B ledger-only`, never added.
- **No ledger rows** (no file, or only `error` rows): `ledger cross-check: no ledger data (...)`.
- **No ledger rows** (no file, or only `error` or load-marker rows): `ledger cross-check: no
ledger data (...)`.

Load-marker rows (`{ t, session_id, loaded: true, cli, mh }`, GH #444, one per module load) carry
no `usd` and feed a separate line before it: `module loaded: N of M sessions since the first load
marker (<date>)`, M being the `costs.jsonl` sessions whose last row is at or after the first
marker, or `module loaded: no load marker (...)` when there is none. A session with spend and no
marker ran with modules off.
Modules were off, the CLI is older than 2.1.287, or no turn has ended yet. Not zero.
- **Error rows** add a `warning:` line with their count; the ledger total may then run low.
- **Rows whose `usd` is not a finite number** (`null`, a string, missing; `null` is how JSON writes
Expand Down
11 changes: 11 additions & 0 deletions skills/meta/cost-report/scripts/cost-report-dedup.js
Original file line number Diff line number Diff line change
Expand Up @@ -90,16 +90,27 @@ console.log("total: "+f4(sum(latest))+" ("+sessionIds.size+" sessions)");
{
const lf=path.join(path.dirname(f),"cost-ledger.jsonl");
const led=new Map();let lerr=0,lbad=0;
// Load markers (GH #444): one `loaded` row per module load, carrying no usd.
const marked=new Set();let firstMark;
if(fs.existsSync(lf))for(const l of fs.readFileSync(lf,"utf8").split(/\r?\n/)){
let r;try{r=JSON.parse(l)}catch{continue}
if(!r||!r.session_id)continue;
if(r.loaded===true){marked.add(r.session_id);if(!firstMark||String(r.t)<firstMark)firstMark=String(r.t);continue;}
if(r.error!==undefined){lerr++;continue;}
// A finite number only (Number.isFinite never coerces): Number(null) and Number("") are 0,
// which would read as a reset and add the next reading whole.
const usd=r.usd;if(!Number.isFinite(usd)){lbad++;continue;}
const p=led.get(r.session_id)||{prev:undefined,total:0};
p.total+=p.prev===undefined||usd>=p.prev?usd-(p.prev??0):usd;p.prev=usd;led.set(r.session_id,p);
}
// Load rate: sessions with spend since the first marker that carry one. A session without one ran
// with modules off (kill switch, --bare, an old CLI); older sessions predate the marker.
if(!firstMark)console.log("module loaded: no load marker (modules off since the marker shipped, or an mh older than it)");
else{
const lastTs=new Map();for(const r of latest){const k=r.session_id||r.transcript_path||r.timestamp,ts=String(r.timestamp);if(!lastTs.has(k)||ts>lastTs.get(k))lastTs.set(k,ts);}
const since=[...lastTs].filter(([,ts])=>ts>=firstMark).map(([k])=>k);
console.log("module loaded: "+since.filter(k=>marked.has(k)).length+" of "+since.length+" sessions since the first load marker ("+firstMark.slice(0,10)+"); one without ran with modules off");
}
if(lerr)console.log("warning: "+lerr+" cost-ledger error rows (the module could not read or write the ledger on those turns) — the ledger total below may run low");
if(lbad)console.log("warning: "+lbad+" cost-ledger rows with no finite usd were skipped (not read as $0) — the ledger total below may run low");
if(!led.size)console.log("ledger cross-check: no ledger data (the mh cost-ledger module writes it; it is off when modules are, on CLIs before 2.1.287, or before a session's first turn ends)");
Expand Down
Loading