Repository navigation

Evaluate sessions through the shared evidence contract - #1160

Merged

realtonyyoung merged 31 commits into

mainfrom

ai-2746-evidence-contract

Sep 25, 2026

Merged

Evaluate sessions through the shared evidence contract#1160
realtonyyoung merged 31 commits into
mainfrom
ai-2746-evidence-contract

Commits on Sep 24, 2026
- Mirror the evidence wire records and keep the harness envelope subtype
  
  realtonyyoung
  and
  claude
  committed
  85c8914
- Omit the evidence hub fields from a legacy PrepareResult/QuestionResultV2
  
  realtonyyoung
  and
  claude
  committed
  a9bd939
- Keep each evidence run in an owner-only directory removed on exit
  
  realtonyyoung
  and
  claude
  committed
  4ef898a
- Write each evidence question's ledger and run file
  
  realtonyyoung
  and
  claude
  committed
  ea94dd0
- Resolve, renew and certify through the wire evidence scope
  
  realtonyyoung
  and
  claude
  committed
  4d3f35b
- Measure evidence coverage from the ledger against the shared fixture
  
  realtonyyoung
  and
  claude
  committed
  31eb4f0
- Drop the plan-coordinate reference from the fixture test's doc comment
  
  realtonyyoung
  and
  claude
  committed
  def5bb3
Commits on Sep 25, 2026
- Render evidence pages and choose the route by a streamed fit test
  
  realtonyyoung
  and
  claude
  committed
  cda13e0
- Nest a resolved deferred tool-call argument under its own call
  
  realtonyyoung
  and
  claude
  committed
  cd018d9
- Build the evidence orientation, retrospective excerpts and prompts
  
  realtonyyoung
  and
  claude
  committed
  d9a37d9
- Report the eval treatment and retain each evidence ledger for baselines
  
  realtonyyoung
  and
  claude
  committed
  3dd02d5
- Serve the eight evidence tools from kcap mcp judge --run
  
  realtonyyoung
  and
  claude
  committed
  35247e0
- Evaluate a single session through the evidence route when advertised
  
  realtonyyoung
  and
  claude
  committed
  c12537b
- Report every uncertified citation and fail closed at the pre-drain check
  
  realtonyyoung
  and
  claude
  committed
  6e0932f
- Run daemon evals through the evidence route inside bounded phases
  
  realtonyyoung
  and
  claude
  committed
  b1272e5
- Judge evidence-route questions with strategy first views and obligations
  
  realtonyyoung
  and
  claude
  committed
  278ed4b
- Pick the reporting question in catalog order and retry a failed first view
  
  realtonyyoung
  and
  claude
  committed
  c967ea8
- Document the evidence route for kcap eval and the daemon
  
  realtonyyoung
  and
  claude
  committed
  ef949f9
- Keep the judge's MCP server serving after a failed tool call
  
  realtonyyoung
  and
  claude
  committed
  b595abd
- Treat a citation answer under another scope version as scope loss
  
  realtonyyoung
  and
  claude
  committed
  5225045
- Tell an evidence-phase timeout from a cancellation by its cause
  
  realtonyyoung
  and
  claude
  committed
  68e2e9c
- State the eval code's comments as present-tense constraints
  
  realtonyyoung
  and
  claude
  committed
  7d42dd2
- Follow the repo's JSON-shape and temp-dir rules in the evidence code
  
  realtonyyoung
  and
  claude
  committed
  93aa9c0
- Fail an evidence read that does not parse or does not advance
  
  realtonyyoung
  and
  claude
  committed
  3344fdb
- Count a citation a successful answer leaves out as dropped
  
  realtonyyoung
  and
  claude
  committed
  e1577c0
- Add the calls a kill left after the last ledger footer to its totals
  
  realtonyyoung
  and
  claude
  committed
  9fc164e
- Keep an evidence run alive until the phase using it ends
  
  realtonyyoung
  and
  claude
  committed
  80eafbf
- Remove only the run a phase's cleanup belongs to
  
  realtonyyoung
  and
  claude
  committed
  f2b68df
- Describe the open_page handle without a model-name lookalike
  
  realtonyyoung
  and
  claude
  committed
  71d57dc
- Resolve null inline arguments from their body and fail an empty paged read
  
  realtonyyoung
  and
  claude
  committed
  ca09fc8
- Drop the case inventories from the evidence test summaries
  
  realtonyyoung
  and
  claude
  committed
  ab6b33f

Commits

Commits on Sep 24, 2026

Commits on Sep 25, 2026