feat(mecatui): correlate root tool hooks in typed scrollback

[jbeda]

Summary

Relates to #2117.

Implement the reviewed root-visible tool-hook correlation data layer:

• preserve exact call and source-event identities at client ingress;
• own hook/review evidence in typed scrollback, with detached Tool/Subagent/Team snapshots;
• attach only to a unique retained root call; keep conflicting/unknown evidence standalone;
• share live/replay reduction, pending-candidate cleanup, and validated approval/detail handoff;
• preserve existing notice visibility, approvals, F9, compact cards, and /toolcalls presentation.

Attached-hook rendering, live-detail persistence, child-session correlation, and historical approval/review joins remain deferred. This PR does not complete #2117.

Verification

• task test
• task lint
• task test:race
• task docs
• go run ./cmd/mecademo (tool call, permission ask/approval, result)
• Focused client/UI/scrollback regression and race tests
• Four-axis panel review with security, architecture, reuse, and duplication; findings fixed and affected axes incrementally rechecked. No open findings.

The first aggregate race attempt timed out under concurrent load; the retry passed.

Plan

The full approved implementation plan is preserved in a verified PR conversation comment.

[jbeda]

Implementation plan archive: 2117-tool-hook-correlation

• Plan source provenance: .scratch/2117-tool-hook-correlation-plan.md
• Slug/branch: 2117-tool-hook-correlation / impl/2117-tool-hook-correlation
• Base: refs/heads/main
• Original branch-start baseline: 599bf013a8e8ff894b30086acd0f12ef3ef447ef
• Observed PR head: 2b6bed609ee043912f44bb3814eb0e239f64d0c3
• Updated: 2026-10-09T23:43:03Z
• Current plan portion SHA-256: e65391f64b0eb70b460ebac5264f2cc2201c7d9389f9b0f10c8c484e821a07cc

Amendment: engine-owned identity simplification

The original client/scrollback data slice is on the observed PR head. The
engine/API identity amendment below is reviewed but not implemented. Joe will
clear the session before separately requesting implementation. This comment is
not a claim that the identity changes or their planned gates are on this PR yet.

The amendment replaces provider-ID correlation with engine-assigned public
ToolCallID, shared HookID/ReviewID identity, ordered presentation revisions and
explicit origin/approval references. Provider pairing IDs stay private. Legacy
loads may assign deterministic session-scoped IDs in memory; no migration-only
summary saves, and normal save points may backfill. Event-log backfill, live
rationale persistence, new rendering and the separate MicroVM flake remain out
of scope. These choices supersede the old no-wire-change scope; original notice,
approval and conservative legacy-evidence requirements remain.

Fresh general/security/architecture plan review found one important pre-record
validation gap; the plan now rejects malformed duplicate provider calls before
RecordAssistant/history mutation, and the targeted recheck resolved it. No
remaining material plan findings. This is design review, not implementation or
runtime certification. Prior data-layer gate results remain separate in the PR
body; the full plan below lists future verification intentions. No content was
redacted or omitted. The original planning artifact is retained locally and its
prior publication remains available in this comment's edit history.

Current plan (verbatim)

#2117: tool-hook correlation with engine-owned identities

Slug: 2117-tool-hook-correlation
Plan path: .scratch/2117-tool-hook-correlation-plan.md
Implementation branch: impl/2117-tool-hook-correlation
Base branch: main (refs/heads/main)
Issue: #2117
Implementation PR: #2239

Amendment and current state

This identity amendment supersedes the original plan's no-wire-change scope and
provider-CallID correlation assumptions. The original six client/scrollback tasks
are implemented in PR #2239; their planning artifact is preserved at
.scratch/2117-tool-hook-correlation/pre-identity-plan.md. Existing notice
visibility, approval ownership, and live-detail limitations remain requirements.
The tasks below are remaining identity work, not a rerun of the completed tasks.

Checkout /home/jbeda/src/mecatl-2 is on the implementation branch at
2b6bed609ee043912f44bb3814eb0e239f64d0c3. Recorded branch-start/base OID:
599bf013a8e8ff894b30086acd0f12ef3ef447ef. Resume this same work here, not in a
nested worktree. Unrelated go.work.sum has eight checksum additions; preserve
them untouched/uncommitted unless Joe settles ownership. Do not update/rebase
main or switch branches as part of this planning request.

The original slice's gates/reviews are recorded in ignored run notes. Its CI
wrapping assertions were fixed in 2b6bed6. The later MicroVM race CI handshake
flake is filed as #2240; do not fold its repair into this amendment or silently
waive the failed gate. Past data-layer gates do not certify the identity work.
No identity implementation is authorized now: review, archive, update HANDOFF,
then stop for Joe to clear the session and separately request implementation.

Goal and settled decisions

Make the API identify the concrete invocation, logical hook/review record,
ordered update, and originating session without clients inferring them from
provider IDs, text, event order, or AskID grammar.

Joe's decisions:

• Engine-assigned public ToolCallID; provider correlation IDs remain private
  replay metadata. Do not add a provider-ID API field.
• Review identity derives from public ToolCallID plus review job.
• One shared record-ID namespace on the existing Hook envelope. Keep configured
  deterministic outcomes and contextual reviews distinct via their payloads;
  no Check abstraction, hook execution redesign, or rename of the Hook API.
• Explicit origin session, ToolCallID, and optional review reference on surfaced
  approval asks and receipts as well as hook evidence.
• Breaking engine/public API changes are acceptable; old stored sessions still
  load and can continue safely.
• No historical event-log backfill. On loading an unchanged legacy session
  snapshot, deterministic session-namespaced IDs may be assigned in memory.
  Do not persist a session summary merely for migration. Existing normal saves
  may backfill IDs and revision metadata.
• Revisions order presentation state; gaps allowed. Not authorization tokens,
  CAS versions, gap-free log offsets, or stronger transactional durability.
  Keep the existing event-log failure policy.

Retain the original root-visible data slice: standalone notices for correlated
and uncorrelated evidence, benign hiding/F9/config, fail-visible warnings,
approval interaction, compact cards and /toolcalls rendering unchanged. No
live rationale/held-result persistence, child hook forwarding, new historical
fetching, event-store redesign, unrelated package/SDK cleanup, or new UI layout.

Grounded existing seams

• engine/session/toolcall.go: call.ID/result.CallID are provider replay pairs;
  ItemID is separate private replay data. Existing persisted capitalized keys
  must continue decoding.
• provider/{openai,anthropic,openaichat}/{stream,request}.go: ingest/replay those
  IDs. engine/agent/loop.go accepts and records calls before dispatch.
• agent/actionreview.go and inboundreview.go: derive current ReviewIDs from
  session/provider-call/job. PrepareAuthorizationContinuation in loop.go can
  re-run action review of the same persisted invocation. Human release changes
  disposition without necessarily re-running the checker; stale-context paths
  normally terminate the action and request a fresh invocation.
• session/{event,session,authorization,kind}.go: calls/results, pending asks,
  private authorization/effective/deferred calls, approvals and relationships.
  adapter/sessnap: embedded calls plus field-by-field authorization DTOs.
  adapter/eventsource: reference log fold; currently cannot reconstruct private
  authorization state or all provider replay fields. Preserve those limitations.
• Run.emitChecked: source sequencing; run-local Seq and RunID are event
  metadata, not an implicit durable record revision across resumed runs.
• internal/adapter/server/{mapper,service,guardrails_status,event_recorder}.go:
  public projection/transcripts, authorization and live detail cache.
  Service.appendEvent alone stamps verified Actor; keep that boundary.
• cmd/mecatui/{client,ui}: canonical hook reducer with local record IDs, review
  and source-event indexes, conflicts and card bindings. Simplify new-format
  identity handling, not visibility or security behavior.
• Trace sibling public boundaries: gRPC, HTTP/SSE, ACP, transcript/session
  inspection, result availability, authorization and delegation parent/child
  projections. A change to Hook.call_id alone is incomplete.

Material identity contracts

Public ToolCallID and private provider identity

Mint a nonempty opaque globally unique public ToolCallID once for each newly
accepted model invocation, before recording, dispatch or related event emission.
Use an existing admitted ID generator. No reuse across turns, sessions, children,
runs or providers. A parked/resumed invocation and its effective argument changes
keep the ID; a newly requested invocation gets a new ID even when the provider
repeats its private pairing ID.

Results (provisional and canonical), asks/authorization, hook/review records,
approval receipts, delegation references and selected inspections carry the
appropriate public ID. Parent delegation and originating child invocation are
separate relationships.

Do not replace private provider pairing fields blindly. Prefer an additive
harness identity internally, preserving the current call/result pair and ItemID
for exact provider replay. Internal names are implementation choices; public
contracts call the new identity ToolCallID. Existing public id/call_id/parent_call_id
fields may change meaning to harness identity instead of adding redundant keys.
Never use a provider pairing ID as fallback new-format public identity.

Public IDs do not repair simultaneously open duplicate provider pairing IDs.
Validate the entire accepted model turn against duplicates within that turn and
any outstanding provider IDs before RecordAssistant, persisted history changes,
card opening, or dispatch. Reject the whole malformed turn through the existing
failure path without recording ambiguous calls or executing tools. A completed
provider ID may repeat in a subsequent invocation. Test both within-turn duplicates
and collisions with restored outstanding calls; a public-ID mint must not mask
invalid private provider pairing.

Shared HookID and ReviewID

Put HookID on the existing Hook envelope. For an ordinary published outcome,
mint an engine ID. Keep silent outcomes silent. If one configured invocation
currently publishes multiple distinct outcome notices (e.g. block and mutation),
preserve separate outcome records/IDs rather than collapsing evidence. HookID
identifies the logical published outcome, not a configured definition or delivery.

For a contextual review:

HookID == ReviewID == derive(ToolCallID, job)

Action, permission and inbound jobs remain distinct. Ordinary and review-backed
ID construction uses disjoint namespaces; clients treat values as opaque and
receive explicit fields rather than parsing derivations. ReviewID may remain as
the name used in review-specific APIs, but denotes the same record, not another
entity requiring a join. If both envelope and nested IDs remain, require exact
equality; mismatch is visible standalone evidence, never an accepted association.

One review record per invocation/job. Re-evaluation after authorization and
approval/disposition updates retain its ID. Prior emitted evidence remains in
history; do not add a speculative review-attempt aggregate.

Record revisions

New-format hook updates include stable HookID and positive monotonic revision.
A session-wide high-water counter is sufficient; no per-hook counter map solely
to obtain ordering. Advance at an authoritative serialized publication path so
concurrent sources cannot emit indistinguishable versions. First ordinary-record
revision need not be one with a session-wide counter.

Store high-water metadata additively in ordinary snapshots. Reconcile it with
committed hook events newer than the snapshot on existing load/recovery paths
before resumed publication, using minimal replay metadata support. Preserve
eventsource's provider/private-state limits. No write-on-read, historical event
rewrite, new summary/save cadence, or migration-only snapshot.

Future versions exceed restored snapshot and committed observed log high-water.
Gaps and current append failure diagnostics/policy remain. No promise to recover
an update absent from both normal snapshot and committed log; no stronger
transaction guarantee or revision-based authorization. Failure to establish
required retained high-water is visible rather than publishing a conflicting
version. Missing legacy revision stays unknown/legacy.

For one HookID: higher revision advances machine state; lower revision cannot
roll it back; same revision and same payload is idempotent; same revision with
different payload/origin/call association is standalone conflict evidence.
Preserve validated live detail/receipts through machine updates. Incomplete
new-format identities cannot overwrite identified records. RunID/Seq remains
source-event metadata/legacy fallback, not primary hook key or cross-run revision.

Explicit origin, asks and receipts

Carry originating SessionID on hooks/review references and surfaced asks/receipts.
Do not parse AskID to find origin. Identify child invocation separately from the
parent delegation invocation; parent transport cannot make a child review attach
to a root tool card. New public IDs are globally unique but origin still governs
authorization and correct placement.

Asks/receipts carry public ToolCallID and optional exact review record ID. Only
explicit matching origin/record/call establishes a historical join for new-format
events. Legacy receipts without that evidence stay standalone; do not infer
historical human verdicts. This adds correlation, not enforcement semantics.

Preserve verified-owner RPC checks, active request tokens, session/review/returned
identity validation, acknowledgement keys, exact AskID receipts and stale/closed
token invalidation. New ID syntax or revisions grants no authority.

Legacy and mixed-history loading

Old snapshots/events decode with absent additive fields; never reinterpret stored
provider IDs as new harness IDs. Preserve existing outstanding AskIDs, legacy
ReviewIDs and acknowledgement/detail keys until those asks settle.

A bounded in-memory legacy-snapshot translation may assign deterministic
session-namespaced ToolCallIDs to retained concrete call occurrences. Pair results
and uniquely matched pending/effective/deferred references to the same ID.
Derivation is stable for the same summary, distinguishes repeated provider IDs,
uses a reserved legacy namespace and reveals no raw arguments/provider IDs.
Prefer ordered occurrence anchors in unchanged retained history; avoid hashing
mutable title/usage/approval state unnecessarily. Do not claim stable synthetic
IDs across arbitrary changes to an unsaved legacy summary.

Once assigned, preserve IDs through effective mutations, compaction archives,
supported fork/copy operations, and normal saves. Normal persisted backfill makes
subsequent reloads use those exact IDs. Loading the same unchanged snapshot twice
yields the same IDs and calls Save zero times. No extra persistence is allowed.

Old event projection can use its own deterministic occurrence anchors where the
existing transcript/replay input proves pairing. Snapshot positions are not log
positions: no guessed cross-source equality, name matching or latest-provider-ID
join. Ambiguous old hooks stay standalone. Continuation events use the translated
pending occurrence's identity; recovery callers carry it explicitly rather than
joining to the historical latest-card index. Test snapshot plus old transcript
adoption together, not only independent decoder roundtrips.

Keep conservative legacy hook reduction when HookID/revision is absent. A bounded
legacy review lookup adapter may preserve old detail/ack keys without inventing a
second new-format record identity. All old sessions load; ambiguous actionable
pairing follows existing fail-closed continuation behavior rather than executing
another invocation or corrupting history.

Client ownership and simplification

Conversation owns one hook record store whether attached or standalone. Use engine
HookID directly for new records: no local-ID allocation or ReviewID-to-HookID map
on that path. Rename stored HookSnapshot to HookRecord (or clarify equivalent
ownership) without two identical structs just for naming. Returned records and
Tool/Subagent/Team snapshots remain detached.

Unique public ToolCallIDs permit the existing narrow card index for new-format
attachment, rather than provider-ID uniqueness scans. Keep necessary ordered
hook-before-call candidates and stable references. Same-ID conflicting associations
fail closed even with an engine uniqueness guarantee; cleanup never removes
already-visible evidence.

Replace original-event history/indexing with HookID/revision logic on the new
path where sufficient. Do not promise a fixed map count: legacy fallback and
standalone contradiction evidence still need idempotent replay. Preserve notices,
approval ownership, clone safety, specialization/results, provider/routing metadata,
revision invalidation and inspector selection. No new attached-hook presentation.

Remaining dependency-ordered tasks

On a fresh implementation request use plan-implement and resume the recorded
branch here. Preserve go.work.sum and existing work. Tasks1–4 integrate before
claiming API correctness; partially translated public/private IDs are not a
finished deliverable. Workers get focused direct-write briefs, no branch switches,
commits or aggregate gates. Integrating agent owns review and completion.

1. Durable invocation identity/provider fidelity. Add accepted-call harness
   IDs and propagate results, effective mutations, asks/authorization/deferred
   siblings and relationships. Preserve exact private replay pairing/ItemID.
   Roundtrip additive fields in sessnap and explicit DTOs; implement deterministic
   legacy translation without saves. Focus session/agent/sessnap/eventsource/provider
   tests on repeated completed vs active duplicate provider IDs, pending resume,
   old snapshot twice/no writes and real provider request projection. Completion:
   new invocation unique, continued occurrence stable, provider replay unchanged.

2. Shared record identity/revisions. Ordinary outcome IDs, derived ReviewIDs
   equal HookID, additive high-water and serialized publication/recovery support.
   Reviewer requests/detail scopes use the same record ID. Tests authorization
   continuation, inbound hold/release/deny, multiple ordinary outcomes, concurrent
   publication, stale snapshot plus event tail and restart. Completion: revisions
   cannot regress retained state; envelope/review/detail IDs agree; no live-detail
   storage expansion.

3. Public contracts/origin projection. Protobuf/server mapper, gRPC, HTTP/SSE,
   ACP, transcript/inspection, results/authorization/delegation/asks/receipts all
   project harness IDs and explicit origins. No public provider-ID field. Generate
   via task generate; update guarded API snapshots with task api:update and engine
   changelog classification per COMPATIBILITY.md. Real caller→wire→client tests,
   child/root collisions, old records and owner-authorized boundaries. Completion:
   no mixed public provider IDs or implicit AskID-origin parsing in changed paths.

4. Canonical reducer/replay integration. Consume record ID/revision/origin,
   remove only obsolete new-path maps/heuristics, retain legacy adapter. Record
   before approval returns. Tests live/replay/reattachment, mixed old transcript
   plus migrated pending snapshot continuation, explicit new receipts, stale/
   mismatched detail, root-child isolation, candidate cleanup, snapshots/results/
   specialization, F9/notices/toolcalls selection and width-normalized visibility.
   Completion: deterministic proven joins and one store, no evidence loss or UI change.

5. Documentation/integration/full review. Update existing owning public pages
   from user-docs/_README.md and architecture/API owners, not new concept pages.
   Finish generation, API/changelog/module proofs and full offline gates. The API/
   persistence scope change warrants a fresh full panel against recorded original
   branch-start 599bf01, whole final candidate including
   uncommitted/new files and amended plan as Spec. Fixes get targeted deltas from
   actual reviewed snapshots; no automatic rebase/merge.

Acceptance and planned verification

Observable checks must cover:

• unique public IDs despite repeated completed provider IDs across turns/sessions/
  children, exact provider replay, active malformed duplicates rejected before effects;
• identity preserved across provisional/canonical results, argument changes,
  authorization/approval pause/resume, reload/recovery, compaction and fork/copy
  (existing copied occurrence versus newly accepted invocation distinguished);
• disjoint ordinary/review namespaces; action/permission/inbound distinct; equal
  envelope/ReviewID; stale/equal-conflicting/incomplete versions cannot overwrite
  machine state, live detail or receipts;
• explicit origin/call/review references end to end; wrong owner/child/stale reply
  cannot attach to root or authorize another ask;
• old snapshot/event fixtures load, deterministic repeated loads with no Save,
  repeated legacy provider IDs distinguished, normal save backfills identities,
  pending/effective/deferred pairing stays correct;
• mixed snapshot/transcript continuation follows explicit proven identity, not
  coincident ordinal or provider-ID equality;
• high-water restoration from snapshot+committed tail, concurrent emission,
  dropped append gaps, cancellation and bounded visible restoration failure;
• historical joins only from explicit new-format references; old receipt/detail/
  acknowledgement behavior remains conservative and authorized;
• all original data-slice notice/clone/specialization/result/selection requirements,
  including width-normalized rendered assertions.

Workers run focused offline root/engine tests with reference adapters and test-owned
stores/UserModelDir. Integrator owns task generate, task api:update, engine changelog,
required standalone/publication proofs (including task test:engine-standalone),
task test, task lint, task test:race, task docs and go run ./cmd/mecademo. Full gates
get600-second timeouts; no live paid providers. Never hand-edit generated contracts.
These are intentions, not results for the unimplemented identity amendment.

Review, PR archive, and clear-session boundary

Run plan-review on this exact amendment with fresh general critique plus focused
architecture/security. Inline it when ignored files are absent in child workspaces.
Record findings/rechecks in .scratch/2117-tool-hook-correlation/identity-plan-review.md.
Material choices remain explicit until answered; review is not build authorization.

Maintain existing PR plan comment via plan-implement publication procedure: verify
PR2239/head branch, marker, author and prior digest; update comment6088059542 in
place, not another dump. Full safe current plan, amendment note, actual baseline/
head provenance, timestamp and plan-portion SHA; read back and verify. Separate
planned gates from old actual evidence. No sensitive omission without disclosure;
failed publication is a visible handoff blocker, not permission to truncate.

After review update ignored root HANDOFF.md with plan/branch/start/current OIDs,
legacy and revision decisions, review outcome/remaining work, PR/comment mapping,
go.work.sum preservation and #2240 flake. Stop there for Joe to clear the session.
No code, commits, pushes, merges, branch changes or unrelated repairs now.