[feat] Show scorer results in the dashboard #1182

Parent: #1137

Goal

Labels on teamai-reports are only worth their cost if someone reads them. Add a scorer view to the existing teamai dashboard that answers two questions: is a scorer's pass rate moving, and does it differ by tool, model, team revision or definition.

What it shows

• One measurement at a time. Before any grouping, the view fixes the scorer, the rubric version (rubric.sourceHash, with teamRevision shown when there is one), the source (workflow or session), the run mode (runMode, [feat] Run mode: tell attended runs from unattended ones #1384) and the judge (judge.kind, and for a model judge its recorded tool and model). Labels that differ in any of these are never summed: a changed rubric, another kind of subject, another run mode or another judge is a different measurement. The view opens on the newest rubric whose labels carry rubric.teamRevision, so a member's uncommitted draft does not become the team's view. When no label of the scorer has one, it opens on the rubric with the latest scoredAt. Labels without judge data form their own judge not recorded cohort.
• Pass rate over a window, 30 days by default, as a headline and as a weekly trend. The window is about when labels were scored, not when the work happened, and the view says so. A week with no labels is a gap, not 0%. The headline shows n, passed and failed next to the rate.
• Group by one recorded configuration dimension: tool, model, team revision, definition, the role, project or tag selection the assets were delivered for (team.selection, [feat] Score a finished session on demand #1178), or fingerprint ([feat] Record the agent each workflow step ran as #1176). Session labels can also be grouped by a team skill the session loaded (used.skills, [feat] Score a finished session on demand #1178), with a label counted under each skill it lists. Model groups keep whether the model was reported or only requested. Labels without that part form a not recorded group, never a guessed one.
• Before and after a team commit. Given a commit of the team repo, the view splits the labels in two: those whose agent.team.revision contains it, checked with git merge-base --is-ancestor in this machine's team clone, and those that do not. A label without team, or whose revision this clone does not have, is not recorded. It combines with grouping by skill, so a team sees the sessions that loaded a skill before and after the commit that changed it.
• Failing labels. Under a cohort, the view lists its failing labels: author, scoredAt, platform and used.skills, newest first. When the team repo holds a pushed session whose origin is the label's subject, the row shows that session's name, so a teammate can open it with teamai session resume or score it with teamai score --team-session. It shows no native session id, prompt or reasoning.
• Pass/fail as stored. passed is read from the label, never recomputed against the scorer's current passingScore.
• A line under the view: Labels from sessions and workflow runs show how the tools did on the work that came along. To compare tools, models or team assets on the same tasks, run a benchmark.

Data

• Workflow labels are read from scores/<member>/<YYYY-MM>.jsonl on teamai-reports, like session labels, told apart by source ([feat] Add workflow scorers and recorded quality labels #1085). For a workflow label, subject.runId, subject.step and subject.pass join it to its run record on teamai-reports, which gives that pass's agent. This view reads labels in [feat] Add workflow scorers and recorded quality labels #1085's shape. A label without rubric.sourceHash goes to a rubric not recorded cohort and is never counted under the current rubric. A label that names no pass, on a step that ran more than once, is shown under its step without a pass, and never attached to an arbitrary one. A record from before [feat] Record the agent each workflow step ran as #1176, with a string agent, gives only the definition.
• Session labels come from scores/*/*.jsonl ([feat] Score a finished session on demand #1178).
• Order. Read every source, keep the newest scoredAt per (scorer, rubric.sourceHash, subject, judge.kind), and only then apply the window, so a label rescored in a later month counts once.
• Benchmarks. Labels from a run whose end record has benchmark ([feat] Run benchmark suites as workflows #1180) are left out. If [feat] Run benchmark suites as workflows #1180 lands later, it adds this exclusion.
• Privacy. The dashboard server answers with Access-Control-Allow-Origin: * (dashboard.ts:300). Its responses carry labels and recorded configuration fields only. No judge reasoning, prompt, transcript or diff is read or returned.

Scope

• Read API and page in the existing dashboard structure, next to the Workflow Runs page ([feat] Add a Workflow Runs page to the dashboard #1088) if it has merged.
• English and Chinese strings.
• docs/guide/dashboard.md and docs/guide/zh-CN/dashboard.md: a section on the scorer view and what it can and cannot show.

Acceptance criteria

1. With fixture labels from two versions of a scorer, the view opens on the newest one whose labels carry rubric.teamRevision, with its own n, and the other shows only its labels. A version whose labels carry no teamRevision does not open by default while another does.
2. Workflow and session labels of the same scorer are never in the same count. Neither are labels from two judge models with the same rubric.
3. Grouping by model puts labels with no model under not recorded, and keeps reported and requested-only models apart.
4. A workflow label is grouped by the agent of the pass it judges. A label on a pre-[feat] Record the agent each workflow step ran as #1176 record has only the definition.
5. A label scored in January and rescored in February counts once, in February.
6. Raising passingScore in the scorer file does not change any past pass rate.
7. A week without labels shows a gap in the trend.
8. No API response contains a key or value from a judge's reasoning file or a session transcript (test with fixture text).
9. Grouping by role puts labels from members with different primaryRole on the same team revision in different groups. Grouping by skill counts a label whose used.skills names two skills under both.
10. A fixture benchmark run with diff-quality labels leaves the view's n for diff-quality unchanged.
11. With labels on revisions A, B (child of A) and C (child of B), splitting at B puts A before and B and C after. A label with no team is not recorded.
12. A failing label whose subject is a pushed session is listed with that session's name and no native session id.

Test plan

• Unit tests for the aggregation: dedup before the window, cohorts, the join to passes, grouping, gaps.
• API tests for criterion 8.
• One real run of teamai dashboard against a fixture reports branch, with a screenshot in the PR.

Out of scope

Benchmark comparisons in the dashboard, per-asset health (#408), judge reasoning in the browser, and alerts.

Dependencies

• Blocked by: [feat] Score a finished session on demand #1178 and [feat] Run mode: tell attended runs from unattended ones #1384. [feat] Add workflow scorers and recorded quality labels #1085 is transitive through [feat] Score a finished session on demand #1178.
• Unblocks: [feat] Open improvement PRs proven by a before/after benchmark #1249, [feat] Compare scorers with hand labels, including on pushed sessions #1269

Update: run modes and session labels

The original scope already covers session labels. This update adds the runMode split (#1384) and a reminder table. Run mode is fixed like source and judge: labels of different modes are never summed, and labels without runMode form their own not recorded cohort.

scorer           runMode       source    labels   passed
review-quality   attended      session   14       71%
review-quality   unattended    workflow  52       88%
review-quality   not recorded  session    9       67%

A member's own dashboard also compares the sessions the friction reminder fired on with those it did not, so the threshold can be tuned with data. "Reminder fired" is hinted in the member's local per-session contribute state (getSessionPath, src/contribute-check.ts:120-128), the learning offer. That state is only on the member's machine, so this table covers only sessions run on that machine, says so, and is not part of the team view.

It compares within one scorer, rubric, source, judge and run mode. A session with no reminder state is not recorded; no reminder state is inferred from a label.

review-quality · rubric 3a9e1c · session · claude / <recorded-model> · attended · sessions run on this machine
reminder fired   sessions   labelled   passed
yes              18         11         64%
no               96         40         85%
not recorded     12          3         —

The state file is swept 24 hours after its session's last write (cleanupStaleSessions, STALE_SESSION_MS, src/contribute-check.ts:183-208), which is shorter than the 30 days window. So the reminder decision must outlive it: when cleanup sweeps a state file, it keeps a record of only hinted, true or false, for every swept file (and the facts #1388 adds to it), removed after 30 days. #1388 needs the same record for its once-per-session offers; whichever of the two issues lands first adds it, and the other reuses it. The table stays local to this machine.

Added criteria:

13. Labels of the same scorer with different runMode are never in the same count; labels without runMode are counted under not recorded.
14. The reminder table reads only this machine's contribute state, and a label whose session has no state there is not recorded.
15. A session whose reminder fired 10 days ago, whose state file was swept since, still counts under yes in a 30 days window.
16. A session whose reminder did not fire, whose state file was swept 10 days ago, still counts under no in a 30 days window.