Skip to content

feat(run-logging): task-event continuum, DVR sessions, permanent trace identity - #1661

Merged
Rod-Christensen merged 21 commits into
developfrom
feat/run-logging
Jul 24, 2026
Merged

Rod-Christensen merged 21 commits into
developfrom
feat/run-logging

Conversation

@Rod-Christensen

@Rod-Christensen Rod-Christensen commented Jul 23, 2026 •

Copy link
Copy Markdown
Collaborator

Run logging & replay: the task-event continuum, DVR sessions, and permanent trace identity

Companion saas PR rocketride-ai/rocketride-saas#360 advances the submodule and hosts the UI; merge this PR first, then the saas PR.

What this is

Today nothing survives a completed task: the engine child's stdout IS the event stream (consumed live, never persisted), and tasks are garbage-collected after a grace period. This branch makes every task write one continuous JSONL event log per projectId.source.runKind — a continuum in which individual runs are just chapter markers — served back through a single rrext_log command, replayed through the existing monitoring panels with a per-source DVR player, and consumed everywhere (live and replay alike) through one client session abstraction. Dev and deployed runs are naturally separate continua.

Architecture, bottom up

L1 — every event carries its identity (9bf99944, hardened in 5fb2ca48, 44a5c680; protocol-corrected in the DAP-fix commit)

  • Every task event's BODY carries the continuum stamps beside its identity: body.eventTime (epoch-seconds float, stamped once where raw stdout frames become JSON) and body.logSeq — the continuum sequence, catalog-seeded (a fresh stream starts at 1; reopen continues at control.lastSeq + 1), strictly monotonic across runs and engine restarts. logSeq is the ordering axis (immune to clock adjustment); time is the human axis.
  • The DAP envelope is pure protocol. seq stays each connection's own monotonic message counter, exactly as DAP requires — an earlier iteration rode the continuum in the header seq, which broke DAP sequencing the moment one connection monitored two tasks (event Y followed by event Y−555). The generic DAP handler (dap_conn.py) is byte-identical to develop: the envelope is transport, the body is the record.
  • Task.build_event assigns logSeq at build time (one seq per delivered event, no gaps from built-but-unsent events); the monitor forward is plain send_event(event, id, body) — the shared body carries the stamps to every subscriber identically.
  • _forward_task_event stamps project_id + source into every task-scoped body centrally, replacing per-event-builder stamping and client-side envelope-id suffix guessing. Why: membership must be a pure body test; identity established at one choke point cannot drift per event type.
  • One representation, no mirror. Consumers read body.eventTime/body.logSeq directly — the SDK LogEvent (TS + Python) and shared-ui's TaskEventMessage have no top-level stamp fields; the body is typed with the stamps required. Legacy segments that carried the stamps at the header are canonicalized into the body at decode (normalizeStamps in both SDK codecs), so old recordings read identically to new and no consumer carries a fallback. The host gates admit an event on one pure body test and pass the message through untouched.
  • Crash semantics: an unpersisted tail may re-issue logSeq values on the next run — accepted, because a crash also drops every websocket, so clients reconnect with fresh sessions and fresh live buckets; nothing stale survives to collide.
  • shell-api was cleared and refrozen at v0 via ./builder shell:freeze (sanctioned: nothing has been released, so no consumer ever saw the pre-fix LogEvent shape). The frozen contract now carries the body-only event type.

L2 — the writer (7feda2a2, 44a5c680)

  • RunLogWriter spools JSONL locally and seals immutable ~16MB segments (24h backstop) uploaded through the user-scoped account FileStore; a fixed 64-slot control file is the time/seq → segment routing table plus the chapters list (run begin/end intervals) that draws the activity bar without touching segments. No compaction, no per-run artifacts, no tokens anywhere in paths or content (tokens are credentials, not names).
  • The log records all event types — it is a log, not a projection; sampling and payload caps bound status volume. Run end force-seals and uploads, so the store holds the full run immediately; the loss window is mid-run only.
  • Recovery is store-side only (K8s reality: the spool dies with the container); stale spool files are swept at startup with strictly pattern-anchored matching.
  • Teardown order: the run log closes last, after the final zeroed status and the terminal task/lifecycle forwards. The log is the record; closing it earlier made every recorded run end without its end.
  • The engine emits an explicit final zeroed status (body.final=true) — clients never infer process death.
  • Schema v2: keyframes + deltas per segment (measured ingest ≈ 30% of raw), codec kept internal (rocketride/_log_codec.py, client/log-codec.ts — not exported).

L3 — one read path (b3086c4c)

  • New rrext_log mixin: chapters (the control file's run intervals with begin/end times + starting seq), read (ranged seq/time reads composing store + spool + in-memory tail, paging cursors, horizon truncation reported honestly), delete, segment (raw fetch for the session codec). Clients never touch storage.
  • Both SDKs gained client.log.* namespaces together, with co-located docs.

DVR sessions — the centerpiece (f0f55c1e)

client.log.openEventStream returns a LogEventStream session (TS client/log-stream.ts, Python rocketride/log_stream.py, mirrored API): seek(pos|'live'), play(pos, speed, cb) (speed 0 = flat-out), pause, getStatus, getTraces, getTrace(beginSeq), getConsole, ingestLive, closeEventStream.

  • Live events are not a special case. Arrivals append to a "live bucket" that is an ordinary cached segment with a sentinel id sorting after every server segment — history walks, trace collection, and status folds traverse disk and the live tail through one code path. Invariant: the live bucket is never discarded. Only prefixes provably server-backed leave it (24MB catalog-reload split, or bytes actually fetched from disk). Any policy that could drop an unbacked live event silently loses data at exactly the moment the user is watching.
  • Two delivery modes, no polling. HISTORY walks the cached segments in order until exhausted, then flips to LIVE; in LIVE an arrival appends to the bucket AND is delivered directly to the callback. The walk cursor dedupes the overlap by seq. Earlier iterations coupled live delivery to disk state and needed poke/poll compensations — this model has no window where an event sits with no deliverer.
  • Segment cache capped at 6 resident segments, LRU eviction (touched on ensure and on serve) — farthest-from-needle eviction would evict exactly what a replay is about to need again.
  • Sessions self-register their monitor (all event classes) on open, refcounted, released on close. Consumers were silently missing OUTPUT/TASK-class events because they depended on whatever the host happened to subscribe; the session knows what it needs.

Trace identity — a contract, not a heuristic (f0f55c1e)

A trace IS its begin event's continuum seq. Pipe slot ids recycle within a run; client fold docIds are synthetic counters — neither survives a refresh and both collide across runs. The begin seq is globally unique for the stream's life, so getTrace(beginSeq) is deterministic from any position, live or replay: binary-search the span table to the owning segment, collect the slot's flow events forward to its end across segments and the live bucket, extend-on-miss for a still-growing tail segment. SSE narration (apaevt_sse by pipe_id) is collected into the same window, and the writer records which segments an in-flight document's SSE touched so a future sparse walk cannot skip SSE-only segments.

UI (6fca0391, 82d4c88d, 44a5c680, 00a03bc6, f0f55c1e)

  • Source-major environment pages: DESIGN | DEVELOPMENT | DEPLOY strip; each source renders one self-contained SourceSection (own pill bar, own panes, own player) bound to its own continuum — sources replay independently, no shared cursor.
  • Unified DVR model: live is not a separate mode — it is the position pinned to now. One position, one ticker; pinned follows the wall clock, unpinned advances at speed × wall clock, paused freezes and unpins. Event arrival never moves the clock (an idle stream still slides dead space past the window); mode is a derived label.
  • useTaskEvents consumes a session: append-only fold with runway throttling (the session plays flat-out, the hook pauses it past cursor + lookahead), fold caps reported honestly. Dead zone = cleared: outside any chapter every pane reports empty/zero at any position — stale panels in the gap between runs read as data from a run that never happened.
  • PlayBar: DVD transport (prev/next track, ±30s, speeds to 25×, GO LIVE), activity bar drawn from chapters alone; the live block is colored from task begin/end announcements (runActive), not timeline-completion guesses, with a square growing edge. |< / >| land on chapter boundaries with an epsilon so a boundary hit resolves into the intended track.
  • Chapter join without fetch: the apaevt_task begin announcement carries beginSeq — the run-begin marker's seq — so clients synthesize the chapter locally and reconcile with the server list by exact equality. A fetch-on-task-begin is both a race (control file not yet rewritten) and a thundering herd (every watching client at once).
  • Trace pane is a pure request list; TraceDetail is a standalone component in a right-docked DetailPanel (persisted width, Prev/Next footer) fetching by beginSeq, rendering the full call tree with Data|JSON|Raw and interleaved SSE narration rows. Analyze's slowest-documents rows open this popup — inspecting why a document was slow is the point of the table.
  • Log pane: capped projection with honesty notices; downloads walk the whole chapter through a throwaway session so the file is complete even when the screen is not.
  • Tool-call tracing: >DBG BEGIN/ENTER/LEAVE/END on the real pipe id renders tool lanes (input/output/duration) in the trace tree.

Riding along

  • 2ad484a4 — shell-api v0 recreated with the SDK client frozen in-bundle, append-only (prerequisite for the shared session surface).
  • 963e86ef — task arguments + debug output move from per-connection engine process flags to per-task pipeline settings applied via .use: they now work identically on local/on-prem/cloud and take effect on the next run, not the next engine restart.
  • 00a03bc6 / 87b15c07 — SourceSection on the stock Card, the bespoke micro-button tier deleted, and its sizing returned as the stock Button mini prop after small proved too large for canvas node cards.

Testing

  • Golden suite test_log_stream.py against real segment fixtures: play() == read() equivalence, splice with no gap/no dup, console/status exact at position, getTrace across recycled slots and segment boundaries, SSE inclusion, active-segment growth, direct live delivery, raced-fetch delivery, the 24MB split, LRU cap.
  • Writer/reader suites: seal boundaries, continuum across runs, chapters, ring + age eviction (store AND spool), recovery matrix, paging, horizon truncation, lease/store-fallback races.
  • Full ai suite green (~1454); tsc clean (shared-ui, client-typescript, vscode); ruff clean. Exercised end-to-end against a live server across multi-run, multi-day streams.

Notes for reviewers

  • Old (pre-keyframe) v1 segments still decode via passthrough. Streams recorded before the final-zero change show gauge carry-forward at track tails forever — data absent by design.
  • Known follow-ups (deliberately out of scope): sparse touched-based getTrace walk, vscode host migration to the session model, streamed chapter downloads, per-registrant monitor routing.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added persistent per-task run logs with live/replay playback and new run-log APIs (chapters/read/segment/delete) plus a LogEventStream experience.
    • Introduced DVR-style PlayBar, new Log and Analyze views, and a refreshed Trace/TraceDetail experience including tool-call lane rendering.
    • Added Task Arguments and Pipeline Debug Output settings and updated Pipeline TTL choices.
  • Bug Fixes
    • Improved separation of diagnostics vs standard output and refined debugger/job-status handling.
  • Breaking Changes / Settings
    • Removed legacy engine/debug-output settings in favor of Task Arguments / Pipeline Debug Output.
  • Documentation
    • Updated VS Code settings, usage, and added run-log API/guide documentation.

Rod-Christensen and others added 12 commits July 21, 2026 00:17
… seq on every task event

Foundation phase of the run-logging plan (per-task JSONL event continuum).
Every task event now carries two header-level fields, stamped server-side
exactly once:

- eventTime (epoch seconds, float): stamped at the stdio ingress where raw
  engine stdout frames become JSON events (TaskDbgStdio.on_event), so it
  reflects emission time, not forwarding time. Derived events (apaevt_flow)
  inherit their source trace event's time. An engine-provided stamp is never
  overwritten, leaving the door open to in-engine stamping later.

- seq: a per-task continuum sequence seeded from the epoch in MICROSECONDS.
  Why epoch-us: a new process's seed always exceeds every seq a previous run
  of the same task identity issued (nothing sustains 1M events/sec), which
  makes seq monotonic across runs, engine restarts, and supervisor crashes
  with no persisted counter handshake; ~1.78e15 stays under JS
  Number.MAX_SAFE_INTEGER. Legacy small per-endpoint DAP counters (from
  build_event) are re-stamped; values at/above CONST_LOG_SEQ_FLOOR (1e14)
  are recognized as continuum seqs and left untouched (idempotent).
  Request/response seq is never rewritten — DAP correlates responses via
  request_seq. raise_log_seq_floor() lets the L2 run-log writer lift the
  seed to max(seed, control.lastSeq + 1) as belt-and-suspenders.

_forward_task_event stamps idempotently as a safety net so every broadcast
path (status updates, SSE, output) is covered by the same single counter.

Transport: plain output lines now preserve their originating channel as the
output category ('stderr' vs 'console') so the run log and Log page can
distinguish engine diagnostics from console prints.

Client folds (shared-ui project module): TraceEvent carries the new required
headers via parseServerEvent, and useTraceState row timestamps are now
event-sourced (eventTime * 1000). Date.now() is ELIMINATED from the fold —
no wall-clock fallback — so the same fold produces identical rows live and
on run-log replay (the L3+ player/DVR depend on this).

New logging constants (segment size, ring, chapters cap, payload caps,
history ages, sampling, read paging) land together in ai/constants.py and
are consumed progressively by later phases.

Part of the run-logging implementation plan (L1 of L1–L5).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…segments, control file, chapters, retention

One continuous log per task identity (projectId.source.runKind): runs are
chapters (lifecycle marker events) INSIDE the stream, never separate files —
which also resolves the deterministic-token collision (successive runs of
the same pipeline reuse the same identity, so per-run files were ambiguous
by construction).

RunLogWriter (new ai/modules/task/run_log.py):
- Append path: filtered event types (output, flow, errors, exit, lifecycle)
  + sampled status snapshots (one per CONST_LOG_STATUS_SAMPLE_SECONDS) +
  per-event payload truncation that always preserves metadata/timestamps.
  Synchronous and store-free — all store traffic is in a background worker.
- Sealing: 16 MB size seal on a line boundary (JSONL segments concatenate
  byte-for-byte) plus a 24 h backstop seal so low-volume long-lived streams
  cap host-loss tail and store lag at <= 1 object/day. Every store object is
  written exactly once and never modified — no compaction, no multipart
  visibility concerns, identical behavior on fs/S3/Azure/memory providers.
- Control file (./logs/{projectId}.{source}.{runKind}.json under the user's
  store scope; token-free paths AND token-free log content): the time/seq ->
  segment routing table and spooled/uploaded location ledger (segments carry
  startTime/endTime/chapterStart/seq/id/state), stream startTime/endTime,
  lastSeq, and the capped chapters list that renders the UI activity bar
  from one small read.
- Retention: 64-segment ring + per-kind history age (dev 7d / deploy 30d);
  eviction deletes store object AND spool copy (lease-deferred) and advances
  the horizon, trimming chapters with it. Long-lived streams never finalize,
  so the ring is their steady state.
- Concurrency disciplines: state flips to 'uploaded' BEFORE any spool
  delete; SegmentLeases defers (never skips) deletes under readers; upload
  failure leaves the spool copy serving reads and retries later.
- Recovery is STORE-SIDE ONLY (K8s container replacement kills the spool):
  reopen trusts 'uploaded' entries, drops 'spooled' ones (the accepted loss
  window), lifts the task's seq floor to lastSeq + 1 over the epoch-us seed
  (belt and suspenders), and records a clock-anomaly lifecycle event in the
  stream itself if the wall clock ever sits below the persisted lastSeq.
- Startup hygiene: sweep_spool_root() deletes stale spool dirs at module
  init — spool state is never salvaged, and dev machines (whose filesystem,
  unlike a replaced container, survives restarts) cannot leak disk.

Task wiring (task_engine.py): writer opens at subprocess start (run-begin
marker doubles as the chapter header; pipeline config hash, trigger, trace
level — never tokens), appends at the _forward_task_event choke point (what
clients see is exactly what replay reproduces), completes the chapter with
its outcome at termination, and records restarts as markers WITHOUT opening
a new chapter (a restart is not a new run). Logging is best-effort: any
failure leaves the task running unlogged.

Tests: 21 new unit tests over the filesystem provider (sealing, backstop,
upload transitions incl. failure/retry, two-run continuum + chapters, ring
and age eviction from both locations, chapters cap, store-side recovery,
clock-anomaly fallback, truncation, leases, spool sweep) + a conftest that
stubs the dist-only bootstrap modules (depends/rocketlib) so the suite runs
against the source tree; real modules always win when present. Full ai
suite: 1389 passed. test_task_engine's bare-Task helper seeds the two new
attributes _forward_task_event now consults.

Part of the run-logging implementation plan (L2 of L1–L5).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…inuum + SDK log namespaces

Server:
- RunLogReader (run_log.py): ranged/paged reads routed via the control file
  (seq/time/segment range forms), each segment resolved SPOOL-first with
  store fallback under read leases held only for the file read (a lazy
  generator would leave lease release to GC timing when a page limit stops
  mid-segment — deterministic release instead). Horizon-honest responses
  (truncatedAtSeq) and cursor paging (nextSeq).
- The ACTIVE (unsealed) segment is now visible to readers: seal-or-continue
  keeps a completed run's tail (incl. its run-end marker) in the active
  spool file, so the control file carries an 'active' descriptor and the
  reader includes it as a virtual segment — the tail is readable the moment
  it is written. end_run closes the file HANDLE (no fd leak) but keeps the
  file + descriptor; the next run in-process reopens and continues it,
  restoring the descriptor so the shared segment keeps one coherent
  start time/seq.
- Recovery unified under one rule — trust the filesystem: 'uploaded'
  entries always kept; 'spooled'/'active' entries kept (uploads re-queued)
  when their spool file exists (same-process reopen) and dropped when it
  died with a previous container (the accepted loss window).
- WRITERS registry: a live stream's reader prefers the writer's in-memory
  control (fresher than the store copy by up to a seal interval), and
  delete routes through the writer's lock so it never races seal/upload.
- delete: beforeTime (segment-granular — drop segments wholly older; trim
  chapters; advance horizon) or all (remove segments + control), both
  locations, lease/ordering disciplines.
- cmd_log.py: LogCommands TaskConn mixin (subcommand table: chapters /
  read / delete) in the cmd_store/cmd_deploy pattern. Streams addressed by
  the plain identity tuple (projectId + source + runKind) — NEVER by token;
  store scoping derives from the AUTHENTICATED user only. task.monitor for
  reads, task.control for delete.

SDKs (both, in sync): client.log namespace — chapters()/read()/delete() —
with full type surfaces (LogChapter, LogActivitySpan, LogChaptersResult,
LogEvent, LogReadResult, LogDeleteResult, LogRunKind) in TS and Python,
plus the co-located guide page (docs/guide/methods/log.md) documenting the
range forms, paging protocol, stamped headers, and permissions.

Tests: 11 reader tests (chapters shape, ranged reads incl. types filter and
time ranges, cursor paging terminates + matches unpaged, horizon
truncation, spool-gone store fallback, delete beforeTime segment
granularity + delete all, live-writer control preference). Full ai suite:
1400 passed.

Part of the run-logging implementation plan (L3 server/SDK of L1–L5).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…er, PlayBar DVD transport, Log + Analyze projections

useTaskEvents (modules/project/hooks) — THE single event-delivery point for
one source's continuum, with media-player semantics per the plan:
(1) DVR ring buffer: live events accumulate (capped, oldest dropped) even
while paused or scrubbing — pause-live/scrub-back/GO LIVE need no refetch;
(2) speed-aware prefetch: replay pages ahead of the playhead via the
host-injected log.read pager, runway scaled by speed; (3) generation-
cancelled seeks: in-flight fetches from a previous position are discarded
on arrival; v1 contiguous-window policy (near seek reuses the buffer, far
seek flushes + refills); (4) live/backfill merge deduped on seq; (5) pacing
decoupled from fetching — dispatch at eventTime deltas / speed from the
buffer. Also detects idle gaps between tracks and exposes the DVD
controller (prev/next track, +/-30s skip, speeds, go-live); PLAY from a gap
auto-skips to the next track's begin. Host-agnostic: readLog is injected,
live events arrive via ingestLive — one instance per source section.

PlayBar (components/play-bar) — the per-source transport + activity bar:
|prev / rewind-30s / play-pause / forward-30s / next| + 1X/2X/4X/10X/25X +
GO LIVE, over the single-lane timeline rendered PURELY from log.chapters
metadata (no segment reads until a seek): filled spans per track (error
tracks red, live span green, minimum span width keeps 3-second runs
clickable), click-to-seek, cursor knob, retention-horizon axis label.

LogPane — the human-readable projection of output/stderr/lifecycle events
(the "Log page is a projection, not a second file" decision), with
tail-pinning that detaches on manual scroll. AnalyzePane — efficiency
analysis over the visible window from server-stamped emission times:
per-component latency (reentrancy-safe enter/leave pairing), per-document
durations, and idle gaps between runs — the trace-analyze concept, now on a
real data substrate.

run_log.py: the active segment opens line-buffered, so every appended event
reaches the OS immediately and rrext_log reads of the ACTIVE segment are
current to the last event — gap-free live composition (store + spool +
active) without a separate in-memory tail structure (the L5 read-v2
mechanism, delivered by construction).

Part of the run-logging implementation plan (L3 UI delivery + L4 player +
L5 pieces of L1–L5).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ward

The monitor forward (MonitorCommands.send_task_event) rebuilt each outgoing
envelope with build_event(), which mints a fresh per-connection DAP seq and
carries no eventTime — so live subscribers never saw the continuum stamps
the run log records, even though every task event is stamped at stdio
ingress. That silently broke everything that correlates live events with
the logged stream: the client replay folds (which have no wall-clock
fallback by design), the live/backfill dedupe seam in useTaskEvents, and
the late-joiner DVR protocol (first live seq X -> read toSeq X-1).

send_event() now takes optional event_time/seq kwargs that restore the
original header stamps onto the built envelope (only as a pair, matching
how Task.stamp_log_event issues them), and the monitor forward passes the
stamps through from the source message. Request/response seq is untouched
— only forwarded event messages are re-stamped, per the DAP request_seq
correlation constraint. Extending send_event (rather than bypassing it to
self.send) keeps one envelope-building path and the existing test seam.

Part of the run-logging plan (L1 stamping, delivery to live monitors).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n unit + Design|Development|Deploy strip

Restructures ProjectView to the final UI direction (v5, source-major),
replacing the five page-level monitoring views (Status/Tokens/Flow/Trace/
Errors) that could only ever show the live task:

- New SourceSection: the self-contained per-source unit. Each section
  carries its OWN pill bar (Status/Tokens/Flow/Trace/Errors/Log/Analyze),
  pane, and PlayBar bound to that source's run-log continuum. Live mode
  renders from host-fed state exactly as before; replay mode derives
  status from the last absolute snapshot in the visible window, folds
  trace/flow through the SAME parseServerEvent/useTraceState path used
  live, and projects Log/Analyze straight off the window. Seeking between
  tracks shows an explicit idle-gap state with previous/next-run actions
  (DVD semantics — PLAY from a gap auto-skips to the next track).

- ProjectView page strip becomes DESIGN | DEVELOPMENT | DEPLOY.
  Development stacks dev-continuum sections; Deploy hosts an optional
  pipeline-level deployLifecycle slot (owned by the deploy feature) above
  deploy-continuum sections. Sources replay independently — no shared
  cursor. Save now lives ONLY on the Design page (inside the canvas);
  the monitoring pages carry no document actions. Persisted view modes
  from before the restructure migrate onto the new strip (old monitoring
  modes -> development). The Development entry carries the live
  error/warning count so problems stay glanceable from any page.

- New host props: liveLogEvents (raw stamped envelopes; sections split
  them per source by body.source or the '<tokenhash8>.<source>' envelope
  id), readLog + fetchTimeline (stream-bound wrappers over client.log),
  and the deployLifecycle slot. traceEvents/onTraceClear are deprecated
  (kept so existing hosts compile) — sections fold trace state
  themselves, which is what makes replay identical to live.

- VS Code webview host wired live-only for v1: accumulates this
  project's stamped events (capped, chunky trim — sections detect the
  shrink and re-feed their seq-deduped buffers) and passes them through;
  no replay bindings yet. The old trace accumulator and trace:clear
  plumbing are gone.

Why source-major: each source is a separate stream/task, and coupling
their players behind one page cursor (v3/v4 designs) made mixed states —
one source live, one replaying — incoherent. One section per source is
also exactly today's multi-status-card mental model, extended.

Part of the run-logging plan (L3-L5 UI delivery).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rozen in-bundle, append-only

Adding the SDK's new client.log namespace broke the shell-api contract:
the frozen v0 bundle inlined a structural copy of RocketRideClient, and
WorkspaceProvider's `client` prop put that copy in an input (contravariant)
position — so ANY additive SDK member read as a breaking change, and
shell:freeze could not mint a successor because its own pre-check compiles
the same failing floor. Design-owner decision (Rod, 2026-07-21): the client
is an MF shared singleton the shell serves to every app, so its API surface
IS part of this contract — keep it inlined in the frozen bundle and make it
append-only, and fix the root cause instead of exempting members.

- WorkspaceProvider no longer takes client/isConnected props: it reads
  both from useShellConnection() (provider-less, ConnectionManager
  singleton — the exact source Shell was reading one level above and
  threading down). Its props are the named, documented
  IWorkspaceProviderProps. With this, the client appears in NO input
  position anywhere on the surface, which is what makes strict value
  floors compatible with additive SDK growth. Rule recorded in
  contract-hold.ts and freeze-shell-api.js: components source the client
  from useShellConnection()/getClient(), never accept it as a prop.

- api.ts exports RocketRideClient as a named type. parseExportedTypes in
  the freeze script now also collects `export declare class` names
  (previously only interface/type — exported classes silently got no
  floor), so the client receives its own covariant per-version type floor:
  Frozen = Current — SDK additions pass, removing or narrowing anything
  ever frozen fails shell-ui's own tsc.

- versions/v0.d.ts recreated (deliberate, review-visible deletion +
  fresh freeze): the new snapshot contains the full client including the
  log namespace, and the slimmed provider props. contract-check /
  index.ts / latest.ts regenerated; the floors remain their plain strict
  form for every member — no exemptions.

Known limits, accepted for now (Rod): npm-release alignment still needs
freeze-at-publish discipline; client member additions do not by themselves
mint a new frozen version (name-set detector); method-parameter bivariance
can pass some parameter narrowings.

Verified: ./builder shell:check green, ./builder shell:freeze idempotent,
full ./builder saas:build green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…verything, chapter self-heal

Large live-e2e iteration batch on the run-logging continuum. Every change
below came out of driving the feature against real runs; each area lists
what changed and why the previous behavior was wrong.

ENGINE / SUPERVISOR (packages/ai)

- Seq assignment happens exactly once per DELIVERED event. Task.build_event
  now stamps the continuum headers itself (the base class's small
  per-endpoint counter is replaced at build time), stdout ingress stamps
  eventTime ONLY, and the derived apaevt_flow is built inside the
  trace-level gate. Why: ingress used to burn a seq on messages that are
  consumed without delivery (apaevt_trace becomes a derived flow event), so
  every visible flow event consumed two seqs and the continuum was riddled
  with gaps.

- Final zeroed status: _terminated's terminal broadcast IS the final status
  (gauges zeroed, body.final=true), emitted after metrics stop and the
  terminal-state assignment, with the run-log closed after it. Why: an
  earlier final-zero was overwritten by a later teardown broadcast, leaving
  charts holding a stale residual reading forever; the stream's last word
  must be the zero. Outcome check reordered (CANCELLED before exitCode==0)
  so user-stops log as 'cancelled', not 'ok'.

- Tool-call tracing: DataConn._tool wraps py_instance.invoke in
  BEGIN/ENTER/LEAVE/END >DBG events via monitorOther('DBG', ...) — the same
  monitor channel the C++ debugger uses — on the REAL pipe id the call runs
  on (caller's open pipe or the borrowed pool pipe; no synthetic ids).
  LEAVE carries result + durationMs + capped output, errors and
  PreventDefault probes included. total_pipes is emitted as 0 and the
  supervisor keeps its current count for such events. Why: control-plane
  tool calls bypassed the pipeflow entirely and were invisible in
  Trace/Flow/log/replay; the recorded durations immediately exposed a
  21s-per-connection network stall that had been misattributed to the
  transport.

- run_log records EVERY event type delivered to clients; the
  LOGGED_EVENT_TYPES whitelist is deleted. Why: it is a LOG — replay and
  post-hoc analysis need what clients actually saw, and a whitelist
  silently drops any newly added event type. Status sampling (final-zero
  exempt) and the per-event payload cap still bound volume.

- Dangling-chapter self-heal: opening a stream closes any chapter left with
  endTime=null as outcome 'interrupted', ended at the stream's last
  recorded activity (capped at the new run's begin). Why: killed/crashed
  runs never write run-end; consumers treated every dangling chapter as
  live (multiple LIVE entries in the run menu) and merged consecutive runs
  into one track.

- Storage via the account FileStore (refuses an empty client_id) with
  .logs/{projectId}/{source}.{runKind}.{seg:06d}.jsonl segments + per-stream
  control file; flat $TEMP spool with pattern-anchored sweep; end_run
  force-seals and uploads so the store holds the full run immediately.
  Monitor forward passes eventTime/seq through so clients see the same
  stamps the log recorded. Tests rewritten for the FileStore layout and the
  record-everything append.

CLIENT DATA MODEL (shared-ui useTaskEvents)

- One DVR: a single seq-ordered buffer and a single position; 'live' is the
  position pinned to the wall clock, replay is the same position moving at
  speed; one transport ticker owns the clock. Why: live and replay were
  separate code paths that diverged (frozen charts, jumped positions,
  trailing dead space skipped on catch-up).

- Shaped reads: statusAt (last snapshot at-or-before position),
  trackEvents (effective track begin -> min(position, end)), chartSeries
  (1Hz grid merging tracks and dead zones), rangeEvents (Analyze brush),
  and trackStats (Now/Avg/Peak/Min scoped to the effective track). Why for
  trackStats: the chart is fixed at a one-minute window, so window-scoped
  stats read zero whenever the run's activity fell outside the visible
  minute; track scope answers what the run actually did.

- Late-joiner backfill: entering live with a log reader fetches the whole
  open track (bounded fallback window until the chapters cache reveals the
  begin, extending backward when it does), deduped on seq, draining
  multi-page continuations on the pinned tick. Why: a view opened mid-run
  had nothing from before its subscribe instant — status kept the chart
  alive but Log/Trace/Flow stayed empty forever.

- Track resolution anchors the synthetic live track at the LATEST run-begin
  marker in the buffer, and chapters are normalized so only the newest may
  be open-ended. Why: with rapid dev restarts several runs die between
  chapter-cache refreshes; anchoring at the first stray event merged them
  all into one track (Trace showed every run combined).

- Seek reuse ladder (serve / extend-left / extend-forward / flush+refill)
  with the anchor clamped to the stream floor. Why: every seek used to
  flush and refetch, blanking all panes for a frame on a one-pixel nudge.

UI (shared-ui components + project module)

- Needle PlayBar: fixed needle, timeline slides underneath, wheel zoom,
  thin-until-hover strip, grab-the-tape scrub, Shift-brush Analyze slice,
  day-grouped run menu with real clock times; transport is
  [Play|Pause] [speed] [|<] [<<] [>>] [>|] [readout] [GO LIVE], with all
  replay-only controls (and GO LIVE itself) hidden while live. Why: the
  previous bar could not navigate hundreds of runs across days, and live
  mode showed controls that had no meaning at the head.

- Status pane: fixed one-minute chart (range toggle removed — wider spans
  are the play bar's job), real clock axis labels ('Now' only when actually
  live), stats footer fed by trackStats, 450px pane body with the footer
  pinned and the chart centered.

- Trace: durations always render as seconds with two decimals (raw float
  millisecond stamps reached the screen before); tool-lane renderers show
  tool name/input and duration/output; scroll containment fixed (the pane
  fills the body exactly and only the tree scrolls — previously two
  scrollbars fought and tall traces were clipped by a non-flex wrapper).

- Log pane: Download Log (text projection) and Download Events (raw JSONL
  of the run) actions; breathing room below the terminal box; empty state
  replaces the frame instead of nesting inside it.

- Uniform EmptyState (the stock shared-ui placeholder) across
  Tokens/Flow/Trace/Errors/Log/Analyze. Why: six panes had six different
  ad-hoc empties; the style guide's control makes them one system. Flow's
  Pipeline/Component toggle hides when there is no data to toggle over.

- Flow pane folds pipeflow.byPipe from the track's flow events at the
  position. Why: status snapshots sample ~5s apart and documents finish in
  under a second, so sampled byPipe was almost always the empty idle
  instant.

Verified: full ai pytest suite green, tsc clean, saas + rocket-ui builds
green, live e2e against real runs (ingest, replay, seeks, tool calls,
multi-run restarts).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…cro-button tier

Stock components exist so every application looks and behaves the same — a
hand-rolled duplicate of a stock component is a defect. This lands the first
conversion of the stock-components adoption plan and removes the style tier
that triggered it.

- PipelineActions: the "Chat now" / "Endpoint Info" buttons are now stock
  <Button small variant="primary|secondary">. The private micro-button tier
  (btnSmBase + primaryBtn/secondaryBtn + the --rr-btn-sm-* sizing, plus the
  unused endpoint-row styles) is DELETED. Why: it was a bespoke button
  system living inside one component; if a smaller button size is ever
  genuinely needed it must be a Button prop, never a per-component style
  block. Clicks stop at the container so canvas node presses never bubble
  into node selection/drag.

- StatusHeader: split into composable pieces — StatusHeaderInfo (state dot +
  name + state + status subtitle) and StatusElapsed ("Started Xs ago") — so
  Card-based hosts can place them in the Card's header/headerActions slots
  instead of nesting a second header row. StatusHeader remains as the
  self-contained composition for non-Card hosts. StatusActions now renders a
  stock <Button small> (danger for Stop, primary for Run/Subscribe) per the
  Card-header rule that header buttons are plain Button small.

- SourceSection: the section shell is now a real stock Card —
  StatusHeaderInfo in the header slot, the action cluster + elapsed in
  headerActions, the pill bar as the Card toolbar, and the pane body +
  PlayBar as an edge-to-edge body (noBodyPadding). Why: the section was
  spreading commonStyles.card and hand-building everything the Card
  component already owns (header row, divider, actions cluster), which is
  exactly how the Endpoint Info button ended up bespoke in the first place.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…pipeline settings

Move "engine arguments" and "debug output" out of the per-connection-group
local-engine settings and re-introduce them as per-task pipeline settings
(taskArguments, pipelineDebugOutput) applied through `.use` at task start.

Why: the old knobs were ENGINE PROCESS flags — changing tracing or adding an
argument meant tearing down and respawning the local engine, and the settings
were meaningless against a cloud connection (no local process to flag). But
both knobs are really properties of a RUN, not of the engine: the engine
already accepts per-task args via `.use`, and --trace=debugOut belongs to the
task being debugged. Modeling them per-task makes them work identically on
local, on-prem, and cloud connections, and takes effect on the next run
instead of the next engine restart.

Details:
- config.ts: taskArguments + pipelineDebugOutput join the pipeline settings
  group (with getTaskArgs()); the local connection group keeps only
  engineVersion. The raw argument string is intentionally passed as a single
  entry — the engine shell-splits it, so quoted paths survive.
- engine-local.ts: the engine now spawns with base args only; no per-task
  flags leak into the process command line.
- debugger/session.ts: launch requests attach getTaskArgs() so F5 runs get
  the same per-task arguments as panel-started runs.
- extension.ts: the engineArgs array->string migration and the old
  local.debugOutput/engineArgs key migrations are deleted with the keys
  themselves — there is nothing left to migrate into.
- package.json/PipelineSettings/SettingsWebview/panels: the settings UI moves
  the two fields into the Pipeline section, adds enum descriptions and
  explicit ordering so the webview and the contributes.configuration model
  stay mirror images of the cloud app manifest.
- shell-ui: SettingSchema documents that `enum` stays string[] per the frozen
  v0 contract while integer schemas (pipeline TTL) may carry numeric entries
  in manifest JSON at runtime — renderers go through String() and coerce back
  via `type`, which is what SettingsPage now does.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…actions use it

The stock-Card migration replaced PipelineActions' private micro buttons with
Button small (26px). That is correct in card headers but overflows the canvas
source-node card: "Drop now" + "Endpoint Info" side by side are wider than
the node. Per the rule that killed the micro tier in the first place — a
smaller button size must be a Button prop, never a per-component style block —
the old sizing (16px tall, 9px font, 0 6px padding, radius 3, weight 500)
returns as a stock `mini` prop, and PipelineActions renders mini in both of
its hosts. `mini` composes after `small` so it wins if both are set.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tity, live/replay parity

One coherent layer replaces the ad-hoc "subscribe + backfill" model: every
consumer of task events now reads from a LogEventStream SESSION over the
run-log continuum, and live delivery is just the last stop of the same walk.

== Client session (log-stream.ts / log_stream.py, both SDKs in lockstep) ==

- Live events are NOT a special case. Arrivals append to a "live bucket" that
  is an ordinary CachedSegment with a sentinel id sorting after every server
  segment, so history walks, trace collection, and status folds all traverse
  disk segments and the live tail through one code path. INVARIANT: the live
  bucket is never discarded — only prefixes proven to be server-backed leave
  it. When the bucket exceeds 24MB the catalog is reloaded and the prefix
  covered by sealed segments is dropped; extending the last disk segment
  advances maxDiskSeq the same way. Why: any policy that could throw away a
  live event that never reached disk silently loses data at exactly the
  moment the user is watching it happen.
- Two delivery modes, no pump/poll machinery: HISTORY walks the cached
  segments in order until exhausted, then flips to LIVE; in LIVE mode an
  arrival appends to the bucket AND is delivered directly to the session
  callback (when pinned + playing + past the watermark). The walk cursor
  dedupes the overlap by seq. Why: the previous arrival-drop/poke iterations
  had windows where events sat in the bucket with no deliverer — the
  two-mode model has none, and disk state is irrelevant to live delivery.
- Segment cache is capped at 6 resident segments, evicting least-recently
  USED (touched on ensure and on pump serve) — farthest-from-needle eviction
  would evict exactly the segments a replay is about to need again.
- Sessions self-register their monitor (all event classes) on open and
  deregister on close, refcounted. Why: consumers were silently missing
  OUTPUT/TASK-class events because the host's canvas-oriented monitor
  (summary/flow) was the only registration; the session knows what it needs
  and must not depend on what some host happened to subscribe.

== Trace identity (contract) ==

- A trace IS its begin event's continuum seq. Slot ids recycle within a run
  and fold docIds are synthetic per-client counters — neither survives a
  refresh, both collide across runs. The begin seq is epoch-microsecond
  seeded and globally unique for the life of the stream, so
  getTrace(beginSeq) is deterministic from any position, live or replay,
  in both SDKs: span-table binary search to the owning segment, then collect
  the slot's flow events forward to its end, crossing segments and the live
  bucket, with extend-on-miss for a still-active tail segment.
- SSE events ride along: the writer records which segments an in-flight
  document's SSE touched, and trace collection includes apaevt_sse rows by
  pipe_id, so node narration ("embedding chunk 3/10") appears inside the
  call tree it belongs to.

== Server (task_engine.py / run_log.py) ==

- _forward_task_event stamps project_id + source into every task-scoped body
  centrally, replacing per-event-builder stamping and envelope-id suffix
  tricks. Why: clients must be able to filter membership with a pure body
  test; identity established at ONE choke point cannot drift per event type.
- apaevt_task begin carries beginSeq — the run-begin lifecycle marker's seq
  (the marker itself is log-only). Clients synthesize the chapter locally
  from the announcement and join the server's chapter list by exact seq
  equality. Why: a task-begin-triggered chapter refetch is both a race (the
  control file may not be rewritten yet) and a thundering herd (every
  watching client fetching at once); carrying the identity in the
  announcement makes the local fix-up exact and free.
- Teardown order: the run log closes LAST, after the final zeroed status and
  the terminal task/lifecycle forwards. Why: the log is the record; closing
  it before the terminal events made every recorded run end without its end.

== UI (shared-ui project module) ==

- useTaskEvents consumes a session instead of listening to host events:
  fold with runway throttling, FOLD_CAP honesty, and DEAD ZONE = CLEARED —
  outside any chapter, every pane reports empty/zero at ANY position, live
  or replay. Why: stale panels in the gap between runs read as data from a
  run that never happened.
- |< seeks to the current chapter's begin (or the previous chapter's end)
  and >| to the chapter end (or next begin), with an epsilon so landing on a
  boundary resolves into the intended track.
- SourceSection owns the session lifecycle, keeps the local timeline fixed
  up from task begin/end announcements (server list reconciled by the slow
  poll only), and hosts the trace DetailPanel: right-docked, width persisted,
  Prev/Next over the request list, TraceDetail fetching by beginSeq.
- Trace pane becomes a pure request list; TraceDetail is a standalone
  component (also opened from Analyze) rendering the full call tree with
  Data|JSON|Raw stock ToggleGroup and interleaved SSE narration rows.
- AnalyzePane's slowest-documents rows open the trace popup instead of
  seeking — inspecting WHY a document was slow is the point of the table,
  and the begin seq each row already carries is the popup's exact key.
- LogPane caps its projection honestly (notices for both the display cap and
  the fold window) and downloads via a full continuum walk, so the file is
  complete even when the screen is not.
- PlayBar colors the live block from task begin/end announcements
  (runActive) rather than timeline completion guesses, with a square live
  edge.

== Tests ==

- test_log_stream.py: golden suite proving play==read equivalence, splice
  and console/status exactness, getTrace by begin seq across recycled slots,
  SSE inclusion, active-segment growth, direct live delivery, raced-fetch
  delivery, the 24MB split, and the LRU cap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 23, 2026 07:23
@github-actions github-actions Bot added docs Documentation builder Node builder tooling and ./builder workflows labels Jul 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor
🤖 Internal: Discord sync marker

Auto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete.

@coderabbitai

coderabbitai Bot commented Jul 23, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 06222d19-76b7-402c-8492-92f212fb14f3

📥 Commits

Reviewing files that changed from the base of the PR and between f6119cb and 5e28b29.

📒 Files selected for processing (3)
  • packages/ai/src/ai/common/dap/transport_stdio.py
  • packages/client-python/docs/index.md
  • packages/client-python/docs/log.md

📝 Walkthrough

Walkthrough

This change adds persistent per-task run-log storage, server log commands, Python and TypeScript replay clients, environment-oriented project monitoring, global pipeline settings, and updated shell, shared UI, and frozen API contracts.

Changes

Run-log continuum and task integration

Layer / File(s) Summary
Segmented storage and task lifecycle
packages/ai/src/ai/modules/task/run_log.py, packages/ai/src/ai/modules/task/task_engine.py
Tasks emit stamped events into append-only JSONL segments with sealing, retention, replay, deletion, and lifecycle handling.
Log routing and validation
packages/ai/src/ai/modules/task/commands/*, packages/ai/tests/ai/modules/task/*
Adds rrext_log commands, task wiring, tool traces, output categorization, and storage/replay coverage.

Client SDKs

Layer / File(s) Summary
Run-log APIs and contracts
packages/client-python/src/rocketride/*, packages/client-typescript/src/client/*
Exposes typed chapters, reads, raw segments, deletion, decoding, and DVR event-stream APIs.
Documentation and exports
packages/client-typescript/docs/guide/methods/log.md, SDK barrel files
Documents and exports the run-log surface.

Project monitoring UI

Layer / File(s) Summary
Event delivery and replay state
apps/vscode/src/providers/ProjectProvider.ts, apps/vscode/src/providers/views/Project/ProjectWebview.tsx, packages/shared-ui/src/modules/project/hooks/*
Forwards stamped task feeds, buffers live events, and folds them into timeline-aware DVR state.
Environment sections and transport
packages/shared-ui/src/modules/project/ProjectView.tsx, packages/shared-ui/src/modules/project/components/SourceSection.tsx, packages/shared-ui/src/components/play-bar/PlayBar.tsx
Replaces legacy tabs with Design, Development, and Deploy pages containing per-source replay sections and transport controls.
Logs, analysis, traces, and rendering
packages/shared-ui/src/modules/project/components/*, packages/shared-ui/src/components/{trace,status,flow,pipeline-actions}/*
Adds log downloads, run analysis, trace details and tool rendering, externally driven status charts, shared empty states, and shared buttons.

Shell and configuration contracts

Layer / File(s) Summary
Pipeline settings migration
apps/vscode/src/config.ts, apps/vscode/src/providers/views/Settings/*, apps/vscode/src/extension.ts
Moves task arguments and debug output to top-level pipeline settings, adds fixed TTL choices, and removes obsolete engine-argument controls and migration handling.
Shell and frozen API contracts
apps/shell-ui/src/*, packages/shell-api/versions/v0.d.ts
Updates workspace provider props, RocketRideClient exports, log API declarations, contract extraction, enum documentation, and OAuth documentation.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

Suggested labels: module:ai, module:ui, module:vscode, module:client-python, module:client-typescript

Suggested reviewers: jmaionchi, anandray, joshuadarron, kwit75, stepmikhaylov

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 70.45% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately captures the main change: run logging with DVR sessions and permanent trace identity.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/run-logging

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements per-task run-log continuum support end-to-end: the engine/server now stamps every task event with stable eventTime + monotonic seq, persists events as segmentized JSONL, serves them via a unified rrext_log command, and exposes DVR-style replay through updated TypeScript/Python SDKs and updated UI consumers.

Changes:

  • Add server-side run-log stamping, persistence, and rrext_log DAP routing (chapters/read/segment/delete) plus supporting tests.
  • Add SDK surfaces (client.log.*, DVR LogEventStream) and internal segment codecs for TS/Python, with co-located docs.
  • Update shared UI + VS Code extension to consume stamped events, support replay-oriented panes (Log/Analyze), and move task args/debug output into pipeline settings.

Reviewed changes

Copilot reviewed 80 out of 81 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
packages/shell-api/versions/v0.d.ts Updates frozen shell-api v0 contract with new log types/API and WorkspaceProvider prop shape.
packages/shared-ui/src/modules/project/utils.ts Adds continuum stamp fields (eventTime, seq) into parsed flow/trace events.
packages/shared-ui/src/modules/project/types.ts Makes trace events carry required continuum headers; updates view-mode model.
packages/shared-ui/src/modules/project/index.tsx Re-exports new DVR/session hooks and panes (Log/Analyze).
packages/shared-ui/src/modules/project/hooks/useTraceState.ts Adds track reset key and uses server-stamped timestamps/trace identity (beginSeq).
packages/shared-ui/src/modules/project/components/LogPane.tsx New log-text projection pane with capped display + full-download support.
packages/shared-ui/src/modules/project/components/AnalyzePane.tsx New analysis pane computing latency/doc spans/idle gaps from stamped events.
packages/shared-ui/src/index.ts Exports PlayBar component from shared UI package.
packages/shared-ui/src/components/trace/renderers/render_tool.tsx Adds renderer for tool-call trace lane (input/output summaries + trees).
packages/shared-ui/src/components/trace/renderers/index.tsx Registers the new tool lane renderer and summary behavior.
packages/shared-ui/src/components/trace/index.tsx Exports TraceDetail alongside Trace/JsonTree.
packages/shared-ui/src/components/tokens/Tokens.tsx Replaces bespoke empty UI with shared EmptyState component.
packages/shared-ui/src/components/status/StatusHeader.tsx Splits header into composable pieces; switches actions to stock Button.
packages/shared-ui/src/components/status/Status.tsx Refactors to pure renderer fed by DVR getters; fixes chart range to 1 minute.
packages/shared-ui/src/components/status/CompletionsChart.tsx Labels/tooltips based on real timestamps; removes parent-driven stat calc.
packages/shared-ui/src/components/pipeline-actions/PipelineActions.tsx Uses stock Button with new mini size; stops bubbling into canvas interactions.
packages/shared-ui/src/components/flow/Flow.tsx Uses EmptyState for “no active …” cases.
packages/shared-ui/src/components/button/Button.tsx Adds mini button size for canvas-node chrome.
packages/client-typescript/src/client/types/log.ts Adds public TS types for run-log chapters/reads/segments + DVR session types.
packages/client-typescript/src/client/types/index.ts Re-exports new log types from the types barrel.
packages/client-typescript/src/client/log.ts Adds LogApi namespace (chapters/read/segment/delete + openEventStream).
packages/client-typescript/src/client/log-codec.ts Adds internal segment codec/decoder for keyframe+delta reconstruction.
packages/client-typescript/src/client/index.ts Exports LogEventStream as public API; keeps codec internal.
packages/client-typescript/src/client/client.ts Adds client.log lazy namespace accessor.
packages/client-typescript/docs/guide/methods/log.md Adds TS SDK guide documentation for run-log APIs and DVR sessions.
packages/client-python/src/rocketride/types/log.py Adds Python TypedDicts/Literals for run-log API types.
packages/client-python/src/rocketride/types/init.py Exports run-log types from the Python SDK types package.
packages/client-python/src/rocketride/log.py Adds Python LogApi namespace (chapters/read/segment/delete + open_event_stream).
packages/client-python/src/rocketride/client.py Adds client.log cached_property for the run-log namespace.
packages/client-python/src/rocketride/_log_codec.py Adds Python reference segment codec (keyframes + shallow deltas).
packages/client-python/src/rocketride/init.py Exports LogEventStream from the Python SDK top-level.
packages/ai/tests/ai/modules/task/test_task_engine.py Updates task test scaffolding for project_id and run-log seq/writer fields.
packages/ai/tests/ai/modules/task/test_run_log_reader.py Adds reader tests for chapters/read/segment/delete + retention/paging coordination.
packages/ai/tests/ai/modules/task/conftest.py Adds standalone-run stubs for runtime-only modules for task tests.
packages/ai/src/ai/modules/task/task_engine.py Implements stamping (eventTime/seq), writer lifecycle, final status semantics, and log append path.
packages/ai/src/ai/modules/task/task_conn.py Registers LogCommands mixin for rrext_log.
packages/ai/src/ai/modules/task/commands/cmd_monitor.py Preserves continuum stamps when forwarding monitor events to clients.
packages/ai/src/ai/modules/task/commands/cmd_log.py Adds rrext_log command router (chapters/read/segment/delete) with permission checks.
packages/ai/src/ai/modules/task/init.py Sweeps stale spool roots on startup.
packages/ai/src/ai/modules/data/data_conn.py Emits tool-call traces via DBG monitor channel with payload capping.
packages/ai/src/ai/constants.py Adds run-log constants for seq floor, segmenting, retention, paging, and keyframe bounds.
packages/ai/src/ai/common/dap/transport_stdio.py Preserves stderr vs stdout category when generating output events.
packages/ai/src/ai/common/dap/dap_conn.py Allows send_event to carry through eventTime + seq for forwarded task events.
apps/vscode/src/providers/WelcomeProvider.ts Stops threading debugOutput/engineArgs in welcome settings payload.
apps/vscode/src/providers/views/Welcome/WelcomeWebview.tsx Updates defaults for new pipeline settings (TTL/trace/args/debug).
apps/vscode/src/providers/views/Settings/SettingsWebview.tsx Extends settings model with taskArguments + pipelineDebugOutput; updates defaults.
apps/vscode/src/providers/views/Settings/PipelineSettings.tsx Updates pipeline settings UX (TTL dropdown + task args + debug output).
apps/vscode/src/providers/views/Project/ProjectWebview.tsx Switches to collecting stamped live events with caps; routes membership by project/body or id suffix.
apps/vscode/src/providers/views/components/panels/OnPremPanel.tsx Removes debug output as a connection option; treats it as pipeline setting.
apps/vscode/src/providers/views/components/panels/LocalPanel.tsx Removes debug output/engine args from connection options; keeps engineVersion only.
apps/vscode/src/providers/views/components/ConnectionConfig.tsx Updates panel wiring for removed connection-level debug/args.
apps/vscode/src/providers/SidebarProvider.ts Passes task args via .use using new ConfigManager accessor.
apps/vscode/src/providers/SettingsProvider.ts Includes new pipeline settings in provider state sent to webviews.
apps/vscode/src/providers/ProjectProvider.ts Passes task args via .use for project runs/restarts.
apps/vscode/src/extension.ts Removes migration logic for deprecated engineArgs/debugOutput keys.
apps/vscode/src/engine/local/engine-local.ts Removes engine-process debug/args flags; keeps server process base args only.
apps/vscode/src/debugger/session.ts Passes task args via debug adapter launch requests.
apps/vscode/src/config.ts Replaces engineArgs/debugOutput with taskArguments/pipelineDebugOutput + new getTaskArgs().
apps/vscode/package.json Removes deprecated engine arg/debug settings; adds pipeline TTL enum + new task args/debug settings.
apps/vscode/docs/usage.md Documents new pipeline execution defaults (TTL choices, task args, debug output).
apps/vscode/docs/installation.md Adds settings docs for taskArguments and pipelineDebugOutput.
apps/vscode/docs/index.md Updates settings reference to match new task argument/debug model.
apps/shell-ui/src/workspace/WorkspaceContext.tsx Stops taking client/isConnected as props; sources connection from useShellConnection.
apps/shell-ui/src/workspace/types.ts Clarifies enum typing/handling for runtime numeric enums.
apps/shell-ui/src/views/settings/SettingsPage.tsx Improves enum default display and numeric enum coercion.
apps/shell-ui/src/contract-hold.ts Documents contract constraints around keeping client out of input positions.
apps/shell-ui/src/contract-check.generated.ts Adds explicit type floors for RocketRideClient and WorkspaceProvider props.
apps/shell-ui/src/components/layout/Shell.tsx Removes prop-threading of client into WorkspaceProvider.
apps/shell-ui/src/api.ts Exports WorkspaceProvider props type and RocketRideClient type as contract surface.
apps/shell-ui/scripts/freeze-shell-api.js Ensures exported classes are floored; documents client contravariance constraint.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
apps/vscode/src/config.ts (1)

233-245: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

Migrate existing task argument and debug settings instead of silently dropping them.

Removing the old per-engine keys while reading only the new global keys makes existing configured arguments/debug output disappear after upgrade.

  • apps/vscode/src/config.ts#L233-L245: add a one-time legacy fallback/migration when the new settings are unset, with defined handling for conflicting development/deployment values.
  • apps/vscode/package.json#L188-L197: retain legacy settings as deprecated schema entries until migration is complete.
  • apps/vscode/docs/index.md#L63-L64: document the upgrade mapping.
  • apps/vscode/docs/installation.md#L45-L46: document the upgrade mapping.
  • apps/vscode/docs/usage.md#L56-L63: document the upgrade mapping and changed global scope.

As per coding guidelines, VS Code extension surface changes must be documented in apps/vscode/docs/.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/vscode/src/config.ts` around lines 233 - 245, Update
apps/vscode/src/config.ts lines 233-245 in refreshConfig to migrate legacy
per-engine task-argument and debug settings when the new global settings are
unset, with explicit handling for conflicting development/deployment values. In
apps/vscode/package.json lines 188-197, retain those legacy settings as
deprecated schema entries. Document the upgrade mapping and changed global scope
in apps/vscode/docs/index.md lines 63-64, apps/vscode/docs/installation.md lines
45-46, and apps/vscode/docs/usage.md lines 56-63.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/vscode/src/providers/SidebarProvider.ts`:
- Around line 531-536: Update the sidebar run call in SidebarProvider’s
client.use invocation to include the same pipelineTraceLevel and ttl defaults
used by editor launches. Reuse the existing configuration/default sources rather
than introducing new values, while preserving the current pipeline, source,
args, and name fields.

In `@apps/vscode/src/providers/views/Project/ProjectWebview.tsx`:
- Around line 170-185: Update ProjectProvider.handleEvent and its monitor
subscription to forward stamped task output and lifecycle events identified by
envelope ID, not only events with body.project_id or summary/flow types. Resolve
the envelope’s task/token source through the project-safe mapping, then post
matching events to the webview so ProjectWebview’s isOurs buffering receives the
complete live log and trace.

In `@packages/client-python/src/rocketride/log_stream.py`:
- Around line 601-618: Update the timed playback branch in the event delivery
loop using _base_pos and _next_event so the playback clock advances before
re-checking pacing after a sleep. Ensure the pending event is not repeatedly
returned with the same delay, while preserving watermark updates, callback
delivery, and speed-zero batching behavior.

In `@packages/client-typescript/docs/guide/methods/log.md`:
- Around line 42-44: Update the getTrace example to pass the trace’s beginSeq
rather than the reused pipe-slot id, and make the corresponding change to the
Python example’s ['id'] argument. Preserve the surrounding trace-drilling
examples and use the beginSeq field from the getTraces summary.

In `@packages/client-typescript/src/client/log-stream.ts`:
- Around line 201-218: Update seek() and the related play() flow to advance
pumpEpoch before any repositioning or awaited seek work, retiring any in-flight
pump before it can resume and mutate basePos/watermark or invoke callbacks.
Ensure play() does not await seek(pos) while the old epoch remains active, and
apply the same ordering to the additional seek/play paths identified in the diff
while preserving normal positioning behavior.
- Around line 525-539: Update the replay-exhaustion branch around nextEvent and
refreshTimeline so events buffered by ingestLive during the asynchronous
transition are drained to the callback before committing to live mode. Preserve
ordering and ensure the live bucket is cleared or otherwise marked consumed,
then set pinned and basePos only after the buffered events have been delivered;
retain pause behavior for completed streams.

In `@packages/shared-ui/src/modules/project/hooks/useTraceState.ts`:
- Around line 140-153: Update the dependency array of the useEffect containing
the resetKeyRef comparison to include resetKey alongside traceEvents and flush.
Preserve the existing reset logic and effect behavior; do not add an eslint
suppression.

---

Outside diff comments:
In `@apps/vscode/src/config.ts`:
- Around line 233-245: Update apps/vscode/src/config.ts lines 233-245 in
refreshConfig to migrate legacy per-engine task-argument and debug settings when
the new global settings are unset, with explicit handling for conflicting
development/deployment values. In apps/vscode/package.json lines 188-197, retain
those legacy settings as deprecated schema entries. Document the upgrade mapping
and changed global scope in apps/vscode/docs/index.md lines 63-64,
apps/vscode/docs/installation.md lines 45-46, and apps/vscode/docs/usage.md
lines 56-63.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: e03f2db8-9e58-4d0a-bbb2-d69eb5891b65

📥 Commits

Reviewing files that changed from the base of the PR and between a438c23 and f0f55c1.

⛔ Files ignored due to path filters (1)
  • apps/shell-ui/src/contract-check.generated.ts is excluded by !**/*.generated.*
📒 Files selected for processing (80)
  • apps/shell-ui/scripts/freeze-shell-api.js
  • apps/shell-ui/src/api.ts
  • apps/shell-ui/src/components/layout/Shell.tsx
  • apps/shell-ui/src/contract-hold.ts
  • apps/shell-ui/src/views/settings/SettingsPage.tsx
  • apps/shell-ui/src/workspace/WorkspaceContext.tsx
  • apps/shell-ui/src/workspace/types.ts
  • apps/vscode/docs/index.md
  • apps/vscode/docs/installation.md
  • apps/vscode/docs/usage.md
  • apps/vscode/package.json
  • apps/vscode/src/config.ts
  • apps/vscode/src/debugger/session.ts
  • apps/vscode/src/engine/local/engine-local.ts
  • apps/vscode/src/extension.ts
  • apps/vscode/src/providers/ProjectProvider.ts
  • apps/vscode/src/providers/SettingsProvider.ts
  • apps/vscode/src/providers/SidebarProvider.ts
  • apps/vscode/src/providers/WelcomeProvider.ts
  • apps/vscode/src/providers/views/Project/ProjectWebview.tsx
  • apps/vscode/src/providers/views/Settings/PipelineSettings.tsx
  • apps/vscode/src/providers/views/Settings/SettingsWebview.tsx
  • apps/vscode/src/providers/views/Welcome/WelcomeWebview.tsx
  • apps/vscode/src/providers/views/components/ConnectionConfig.tsx
  • apps/vscode/src/providers/views/components/panels/LocalPanel.tsx
  • apps/vscode/src/providers/views/components/panels/OnPremPanel.tsx
  • packages/ai/src/ai/common/dap/dap_conn.py
  • packages/ai/src/ai/common/dap/transport_stdio.py
  • packages/ai/src/ai/constants.py
  • packages/ai/src/ai/modules/data/data_conn.py
  • packages/ai/src/ai/modules/task/__init__.py
  • packages/ai/src/ai/modules/task/commands/cmd_log.py
  • packages/ai/src/ai/modules/task/commands/cmd_monitor.py
  • packages/ai/src/ai/modules/task/run_log.py
  • packages/ai/src/ai/modules/task/task_conn.py
  • packages/ai/src/ai/modules/task/task_engine.py
  • packages/ai/tests/ai/modules/task/conftest.py
  • packages/ai/tests/ai/modules/task/test_log_stream.py
  • packages/ai/tests/ai/modules/task/test_run_log.py
  • packages/ai/tests/ai/modules/task/test_run_log_reader.py
  • packages/ai/tests/ai/modules/task/test_task_engine.py
  • packages/client-python/src/rocketride/__init__.py
  • packages/client-python/src/rocketride/_log_codec.py
  • packages/client-python/src/rocketride/client.py
  • packages/client-python/src/rocketride/log.py
  • packages/client-python/src/rocketride/log_stream.py
  • packages/client-python/src/rocketride/types/__init__.py
  • packages/client-python/src/rocketride/types/log.py
  • packages/client-typescript/docs/guide/methods/log.md
  • packages/client-typescript/src/client/client.ts
  • packages/client-typescript/src/client/index.ts
  • packages/client-typescript/src/client/log-codec.ts
  • packages/client-typescript/src/client/log-stream.ts
  • packages/client-typescript/src/client/log.ts
  • packages/client-typescript/src/client/types/index.ts
  • packages/client-typescript/src/client/types/log.ts
  • packages/shared-ui/src/components/button/Button.tsx
  • packages/shared-ui/src/components/flow/Flow.tsx
  • packages/shared-ui/src/components/pipeline-actions/PipelineActions.tsx
  • packages/shared-ui/src/components/play-bar/PlayBar.tsx
  • packages/shared-ui/src/components/status/CompletionsChart.tsx
  • packages/shared-ui/src/components/status/Status.tsx
  • packages/shared-ui/src/components/status/StatusHeader.tsx
  • packages/shared-ui/src/components/tokens/Tokens.tsx
  • packages/shared-ui/src/components/trace/Trace.tsx
  • packages/shared-ui/src/components/trace/TraceDetail.tsx
  • packages/shared-ui/src/components/trace/index.tsx
  • packages/shared-ui/src/components/trace/renderers/index.tsx
  • packages/shared-ui/src/components/trace/renderers/render_tool.tsx
  • packages/shared-ui/src/index.ts
  • packages/shared-ui/src/modules/project/ProjectView.tsx
  • packages/shared-ui/src/modules/project/components/AnalyzePane.tsx
  • packages/shared-ui/src/modules/project/components/LogPane.tsx
  • packages/shared-ui/src/modules/project/components/SourceSection.tsx
  • packages/shared-ui/src/modules/project/hooks/useTaskEvents.ts
  • packages/shared-ui/src/modules/project/hooks/useTraceState.ts
  • packages/shared-ui/src/modules/project/index.tsx
  • packages/shared-ui/src/modules/project/types.ts
  • packages/shared-ui/src/modules/project/utils.ts
  • packages/shell-api/versions/v0.d.ts
💤 Files with no reviewable changes (2)
  • apps/vscode/src/providers/WelcomeProvider.ts
  • apps/vscode/src/extension.ts

… drain, epoch races, vscode event routing

Fixes for the verified review findings on the run-logging PR. Each carries the
why; two findings were rejected (reasons on the PR threads: the conftest stubs
are session-scope shims by design, and migrating engine-process args into
per-task args would inject semantically wrong flags).

- log-stream.ts / log_stream.py: timed playback (speed > 0) stalled forever on
  the first inter-event gap — the pacing branch slept/rescheduled and then
  recomputed the SAME delay from an unchanged playback clock. The clock now
  advances past the wait (epoch-guarded in TS so a retired timer cannot touch
  the live clock). The review caught the Python copy; the TS setTimeout
  re-entry had the identical stall and is fixed to match.

- log-stream.ts / log_stream.py: an event arriving during the history-walk's
  final awaits (nextEvent exhaustion -> refreshTimeline) was buffered into the
  live bucket but never delivered — ingestLive only delivers arrivals AFTER
  the pin flips, and the walk was already done. The pump now synchronously
  drains bucket events above the watermark right after pinning; no await can
  interleave an arrival mid-drain in either runtime.

- log-stream.ts: seek() now retires any in-flight pump (pumpEpoch bump) before
  its repositioning awaits — stopTimer only kills a SCHEDULED re-entry, so a
  pump parked on a fetch could resume against the old epoch and mutate
  watermark/basePos or fire a stale callback interleaved with the new
  position. Pump post-await guards also re-check playing/closed, so a pause
  landing mid-fetch cannot leak one extra delivery. (Python is immune: its
  seek cancels the pump task outright.)

- vscode ProjectProvider/ProjectWebview: the host monitor subscribed only
  summary/flow and forwarded only status/flow events, so the webview's live
  log and run-active timeline never received output/lifecycle events. The
  monitor now adds output+task, handleEvent forwards every stamped event for
  the editor's project, and the webview's membership check becomes the pure
  body.project_id test (the envelope source-suffix hack is deleted — the
  server's central identity stamp made it obsolete).

- vscode SidebarProvider: sidebar-started runs now pass the same
  pipelineTraceLevel + ttl as editor launches — a pipeline must run
  identically regardless of where it is started.

- shared-ui utils.ts: parseServerEvent skips flow events missing the continuum
  header stamps instead of emitting undefined eventTime/seq into the folds
  (NaN timestamps, unstable ordering). A skip, not a wall-clock fallback: the
  folds are event-time-sourced by design; against a pre-continuum server the
  trace pane is empty rather than corrupt.

- useTraceState.ts: the fold-reset effect reads resetKey but did not list it —
  the reset only fired because both callers change the traceEvents reference
  alongside it. The dep is added; the ref compare keeps re-runs idempotent.

- docs guide/methods/log.md: both getTrace examples passed the recycled pipe
  slot (.id) instead of the trace identity (beginSeq), contradicting the
  contract stated in the same file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 23, 2026 16:22
@Rod-Christensen

Copy link
Copy Markdown
Collaborator Author

Review findings addressed in 41334c9 — all nine inline threads are replied to and resolved (7 fixed, 2 declined with reasons on the threads).

On the outside-diff finding (config.ts: migrate legacy engineArgs/debugOutput instead of dropping them): declining the automated migration, deliberately. The old and new settings are not semantically equivalent: local.engineArgs were engine process flags (spawn arguments for the local engine), while taskArguments are per-task arguments passed to every run via .use on any connection. Silently copying engine-process flags into per-task arguments would inject flags the task runner never expected — a worse upgrade experience than the settings simply reverting to defaults. debugOutput alone would migrate cleanly, but it lived per-connection-group (development/deployment could disagree) while the new setting is global, so any migration has to invent a conflict rule for a niche path. Users who had these set will find both settings in the Pipeline section of the same settings UI. The removal and the new per-task model are documented in apps/vscode/docs/usage.md.

Beyond the reported findings, one bonus fix rode along: the TypeScript pump had the same timed-playback stall CodeRabbit found in the Python copy (its setTimeout re-entry recomputed the identical delay from an unchanged clock) — both SDKs are fixed in lockstep.

Validation: full ai suite green, tsc --noEmit clean (shared-ui, client-typescript, vscode), ruff clean, shell-ui + rocket-ui bundles rebuilt.

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 80 out of 81 changed files in this pull request and generated 2 comments.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/shared-ui/src/modules/project/utils.ts`:
- Around line 53-58: Update the guard in the fold/event processing utility to
require finite numeric values for both msg.eventTime and msg.seq, using
Number.isFinite rather than typeof checks alone. Apply the same finite-value
validation in the ProjectWebview live-event gate, preserving the existing
behavior of returning or skipping invalid events.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: f6c1c9d6-fb89-47fb-b67f-6a9c2bed39af

📥 Commits

Reviewing files that changed from the base of the PR and between f0f55c1 and 41334c9.

📒 Files selected for processing (8)
  • apps/vscode/src/providers/ProjectProvider.ts
  • apps/vscode/src/providers/SidebarProvider.ts
  • apps/vscode/src/providers/views/Project/ProjectWebview.tsx
  • packages/client-python/src/rocketride/log_stream.py
  • packages/client-typescript/docs/guide/methods/log.md
  • packages/client-typescript/src/client/log-stream.ts
  • packages/shared-ui/src/modules/project/hooks/useTraceState.ts
  • packages/shared-ui/src/modules/project/utils.ts

Copilot AI review requested due to automatic review settings July 23, 2026 17:17
@Rod-Christensen
Rod-Christensen merged commit 0c08ea4 into develop Jul 24, 2026
28 checks passed
@Rod-Christensen
Rod-Christensen deleted the feat/run-logging branch July 24, 2026 02:41
joshuadarron added a commit that referenced this pull request Jul 29, 2026
PR #1661 rewired every accumulating pane — Trace, Errors, Status, Tokens,
Flow, Log, Analyze — to read through a DVR session obtained from the
host's `openEventStream`, a prop no host in this repo binds. Hosts that
have not bridged it across their transport (the VS Code webview is one)
passed a null session, `useTaskEvents.ingestLive` dropped every event, and
the panes rendered their empty states mid-run with no error anywhere.

Fall back to an in-memory store over the live events the host already
delivers. It satisfies the same session contract `deliver` expects — a
pure append, each event handed over exactly once, oldest first, across
repeated `play` calls — and is keyed per stream identity so the several
views `SourceSection` opens all share one buffer. Replay of recorded runs
still requires the real stream.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
joshuadarron added a commit that referenced this pull request Jul 31, 2026
* feat(nodes): add tool_pipedrive CRM tool node

Exposes the Pipedrive REST API v1 to agents: 256 tools across 24 resource
groups (deals, persons, organizations, activities, pipelines, stages, notes,
leads, products, fields, files, users, roles, permission sets, teams, goals,
filters, webhooks, subscriptions, mailbox, call logs, projects, misc), plus a
generic `request` tool that reaches any remaining v1 endpoint through the same
auth, rate-limit and read-only layer.

Full coverage is more tools than an LLM can choose between reliably, so
`pipedrive.toolGroups` controls which groups are published. The default eight
core CRM groups publish 108 tools; `["all"]` publishes everything. Filtering
happens in `IInstance._collect_tool_methods()`, so a gated tool is invisible to
`tool.query` and refused by `tool.invoke` alike.

Client details specific to Pipedrive:
- `{"success", "data", "additional_data"}` envelope is unwrapped; `success:
  false` on an HTTP 200 is raised as an error.
- `x-ratelimit-reset` is the seconds remaining in the window, not an epoch
  timestamp as in the GitHub client this otherwise mirrors.
- Offset pagination (`start`/`limit`, max 500) everywhere except projects,
  which use cursor pagination.
- Personal API tokens go in the `api_token` query parameter; JWT, long or
  `Bearer `-prefixed values go in an Authorization header.
- Custom fields are grouped under `custom_fields` on read and written through
  the `extra` passthrough that every create and update tool accepts.

Tests: 81 unit tests that need no credentials or network, plus an env-gated
live suite behind PIPEDRIVE_API_TOKEN, with record-creating tests additionally
behind PIPEDRIVE_ALLOW_WRITES.

Also corrects the stale node/service counts in docs/README-nodes.md
(88 -> 118 directories, 118 -> 148 services).

Closes #1675

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes): use the real Pipedrive brand mark for the tool_pipedrive icon

The first icon was a green uppercase "P" next to an unrelated dark dot. The
actual Pipedrive mark is a solid green disc carrying a white lowercase "p"
whose counter shows the disc through it.

Redrawn as plain geometry (disc r=128, stem 36x150, bowl r=56, counter r=27)
so it stays crisp at 16px with no gradients, clip paths or embedded raster.

Uses the default nonzero fill rule with the counter arc wound opposite to the
bowl. Under evenodd the stem and bowl would XOR where they overlap and punch a
green notch out of their junction; nonzero merges them solid while the opposing
winding still renders the counter as a hole.

Keeps the viewBox (SVGR runs with dimensions:false, so an icon without one
computes to zero width) and keeps two distinct colours, which makes the
auto-currentcolor svgo plugin treat it as a multicolour brand mark and pass the
authored colours through untinted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes): trace the tool_pipedrive icon from the supplied brand artwork

The previous version approximated the mark with circles. The real letterform
differs in three ways that are visible even at 16px: the stem carries a spur at
the top left, the bowl is an ellipse rather than a circle, and the counter is an
ellipse tangent to the stem's right edge.

Geometry was measured off the supplied 272x272 artwork rather than eyeballed —
connected-component analysis to isolate the counter, then a least-squares fit of
the bowl ellipse to the glyph's right-hand edge (max residual 2px). Rasterising
the traced primitives and diffing against the source leaves 1.47% of opaque
pixels differing, all of it a sub-pixel outline along the shape boundary.

Still a two-colour multicolour mark, so the auto-currentcolor svgo plugin passes
the authored colours through, and still nonzero winding with the counter wound
opposite the bowl so the stem and bowl merge solid while the counter stays open.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(nodes): warn when tool_pipedrive publishes too many tools for an agent

Full coverage is 256 tools. An operator can already reach that by selecting
groups, and nothing told them the agent's tool choice degrades long before it.

Adds a guard rail that counts published tools rather than groups, because group
sizes are lopsided: deals is 28 tools and permission_sets is 3, so a group count
bounds nothing useful. The threshold is 120, just under the 128-function cap
some providers impose on a single request and above the 107 the default groups
publish.

Crossing it warns and carries on. The tools are still published, in both
validateConfig (so it shows in the config panel while editing) and beginGlobal
(so it shows at pipeline start). Silently dropping tools an operator asked for
surfaces later as "the agent won't do X", which is worse than a noisy panel. The
warning is deliberately not raised from _collect_tool_methods, which runs on
every tool.query and tool.invoke.

"all" is exempt: it is already an explicit opt-in to the whole surface, and
scripted callers that never route through an LLM legitimately want it. Listing
every group by name is not exempt.

Group sizes are derived from the __pipedrive_group__ stamps rather than a
hand-maintained table, so a new tool cannot drift out of sync; a test asserts
every group in ALL_GROUPS has a count. Another test pins the advisory semantics
so the warning is not later "fixed" into a silent truncation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai): flatten provider content blocks to visible text

Anthropic with extended thinking returns `content` as a list of typed
blocks, not a string. Every non-streaming path handed that list straight
to callers, who stringified it — so the answer arrived as the blocks'
repr, carrying the raw reasoning and its base64 signatures.

For agents it was worse than cosmetic: the ReAct parser reads output back
with a greedy `Action Input: (.*)`, so the serialized wrapper's trailing
`", "type": "text"}]` landed inside the tool arguments and every declared
field then reported as missing.

Extract the block vocabulary that only the streaming path knew into
`ai.common.utils.flatten_content_blocks` and route chat's invoke, the
stream fallback, and the agent's `extract_text` through it. The streaming
path now shares that vocabulary instead of duplicating it, so the paths
cannot drift; the fallback also forwards recovered reasoning to
`on_reasoning_chunk` rather than dropping it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes): make CrewAI tool calls and hierarchical kickoffs survive

Three defects made the CrewAI nodes fail on runs that should have worked.

Tool arguments were dropped. `_ToolInput` declared a single `input` field,
and CrewAI's argument filter intersects the model's emitted keys with the
schema's property names — so every real parameter was discarded before
validation, which then reported all required fields missing. Generate a
typed model per tool instead: JSON Schema types map to real annotations
(so pydantic coerces "123" -> 123), optional fields accept an explicit
null, and paramless tools get an empty schema. The JSON schema is no
longer appended to the description either; CrewAI renders args_schema into
the prompt itself, and the two copies disagreed.

`HostInvokeLLM` never declared `supports_function_calling()`. CrewAI
defines it on `LLM` but not on `BaseLLM`, and several call sites invoke it
unguarded — converter.py, tool_usage.py, reasoning_handler, and
task_evaluator. The AttributeError was swallowed by converter's broad
except and re-reported as "Failed to convert text into a Pydantic model".
Declare it False: the host channel is text-in / text-out and cannot honour
native tool calls, so every call belongs on the ReAct path. Pin crewai <2
for the same reason — 1.15.x moves the BaseLLM contract.

Hierarchical crews then died in planning. `Crew(planning=True)` runs a
task with `output_pydantic` set and drives it with the host wrapper, which
cannot promise JSON; unparseable plan text dead-ends the converter and
raises "Failed to get the Planning output". Make planning opt-in via a new
**Crew Planning** field, off by default.

Also clear `Task.tools` after construction: CrewAI back-fills them from
the task's agent, and a hierarchical crew resolves the executing agent's
toolset for the manager — which let the manager work a delegate's tools
itself instead of delegating.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes): move tool_pipedrive search to the v2 API

Pipedrive retired the v1 search routes: `/persons/search` and its siblings
answer 404 "Unknown method ." and `/itemSearch` answers a plain 404, so
every search tool the node published was dead. Non-search v1 endpoints
still route normally, so only the search calls move — a blanket swap would
break the many endpoints with no v2 equivalent.

Adds a v2 base URL (company-domain aware, sharing v1's normalisation) and
a v2 envelope wrapper: v2 replaced offset pagination with an opaque
`next_cursor`, which `paginated` would have missed entirely and reported
as a single page. Search tool schemas now advertise `cursor` instead of
`start`, and `item_search_by_field` follows v2's renames — `field_type` ->
`entity_type` with the "Field" suffix dropped from each value, and
`exact_match` replaced by the three-way `match` mode.

Also turns Tool groups into a multi-select dropdown with per-group tool
counts, so the field no longer requires typing group names from memory.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(shared-ui): keep live panes working without a host event stream

PR #1661 rewired every accumulating pane — Trace, Errors, Status, Tokens,
Flow, Log, Analyze — to read through a DVR session obtained from the
host's `openEventStream`, a prop no host in this repo binds. Hosts that
have not bridged it across their transport (the VS Code webview is one)
passed a null session, `useTaskEvents.ingestLive` dropped every event, and
the panes rendered their empty states mid-run with no error anywhere.

Fall back to an in-memory store over the live events the host already
delivers. It satisfies the same session contract `deliver` expects — a
pure append, each event handed over exactly once, oldest first, across
repeated `play` calls — and is keyed per stream identity so the several
views `SourceSection` opens all share one buffer. Replay of recorded runs
still requires the real stream.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(test): stop node packages being shadowed by same-named test dirs

nodes/test is a package whose subdirectories are named after node packages
(text_output/, response/, image_cleanup/, telegram/, tool_git/, ...). Two places
put that directory on sys.path so they could bare-import a test-root module:
conftest (for _sys_modules_guard) and cloud_tts/test_cloud_tts (for
test_contracts). Once nodes/test sits ahead of src/nodes, `import text_output`
resolves to the *test* package, whose __path__ holds no node module — so
`from text_output.instance import Instance` dies with ModuleNotFoundError.

Whether it bites was pure collection order. The affected modules bootstrap with
"insert src/nodes only if absent", which never re-prioritises: previously nothing
seeded src/nodes before cloud_tts (c) inserted nodes/test, so image_cleanup (i)
was the first to insert src/nodes and landed it in front. The new agent_crewai
tests seed src/nodes during collection of agent_crewai (a) instead, so cloud_tts
now stacks nodes/test on top of it and the guard short-circuits in all four later
modules — four collection errors, none of them in code this branch touches.

Import both test-root modules package-relative and drop the sys.path inserts, so
nodes/test never shadows a node package. cloud_tts gains the __init__.py the rest
of the node-named test dirs already have, which is what makes the relative import
resolvable. The four bootstraps now move src/nodes to the front unconditionally
rather than skipping when it is already somewhere on the path, and the guard suite
pins both invariants: nodes/test stays off sys.path, and a node package imported
by name resolves under src/nodes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes, shared-ui): address CodeRabbit review on the tool_pipedrive PR

Security
- pipedrive_client._company_host accepted anything up to the first slash, so a
  configured domain of "evil.example#" built https://evil.example#.pipedrive.com
  and Requests sent it — with the api_token or Authorization header attached — to
  evil.example. Cut at the first path/query/fragment delimiter, refuse userinfo
  and ports, lower-case, and require a single DNS label; anything else falls back
  to the generic host.
- Free-form ids (lead uuids, permission-set ids, channel ids, goal ids, note
  comment ids) were f-stringed into request paths unencoded, so a value carrying
  "/", "..", "?" or "#" retargeted the call at another endpoint — for the DELETE
  tools, a destructive one. Added _base.path_segment() and applied it at all 20
  string-id sites.
- Pin requests>=2.34.2 to match constraints.lock, as tool_cognee/tool_github/
  tool_n8n already do.

Correctness
- crewai_base._annotation_for passed prop['type'] straight to dict.get. JSON
  Schema spells a nullable field as ["string", "null"], and an unhashable key
  raises TypeError instead of returning the default — one such property aborted
  the whole _build_crew_tools loop. Non-str types now fall back to Any, which is
  what the docstring already promised.
- deal_followers_users_list GET the same /deals/{id}/followers as
  deal_followers_list and only differed by running follower rows through
  clean_user, emitting a user-shaped object built from mismatched fields.
  Dropped it; resolving followers to users needs a /users/{id} call per follower.
- add_time (deals, organizations, persons) and origin_id (leads) were in the
  write-key tuples but not in the matching schema props, so body_from could never
  populate them. Added the properties.
- product_variation_update and both project-plan updates passed the builtin dict
  as the response cleaner: dict(None) raises TypeError on an empty 204 body.
  Switched to passthrough, as every other write already uses.
- liveEventSession.getStatus scanned from the live edge, contradicting the
  documented "as of the session position" and the caller, which seeks to an
  anchor first — a repositioned view seeded a replayed run with the current
  status. Scan back from the watermark.

Maintainability
- Eight bulk deletes each repeated the ids guard, the CSV build and a trailing
  _require_write(), and left ids optional so a missing argument surfaced as a
  hand-rolled ValueError. Added _base._delete_bulk (write gate first), marked ids
  required in every schema, and delegated all eight.
- projects._cursor_list re-implemented paging_params_v2 and the v2 envelope
  shaping; it now uses the shared helpers, so the cursor contract has one
  implementation instead of two that can drift.
- file_list advertised the shared 1-500 range, but /files documents a maximum
  limit of 100. PAGING/paging_params/_list take a max_limit override and file_list
  passes 100.
- _build_crew_tools' docstring still claimed the JSON Schema is embedded in the
  tool description, which stopped being true when args_schema took over.
- Tests: 'ACME'.lower() evaluated to 'acme' before the call, so the case path was
  never exercised; test_tools imported pipedrive_client bare, giving the module a
  second identity in a run that also collects test_pipedrive; a vacuous
  b64encode assert, a missing empty-pipelines guard, and a hard-coded field_id
  of 1 that fails rather than skips on a fresh sandbox.

Two findings were not applied. `extra` is not dropped by fields/misc — body_from
merges args['extra'] by default (tools/_base.py). And the @patch/parametrize
signatures are already correct: mock injects positionally innermost-first while
pytest passes parametrize values by keyword, so the suggested reorder would bind
the mocks to the parametrized names.

New tests cover the rejected domains, path-segment encoding, non-str schema
types, the write-gate ordering, and the watermark-bounded status.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes): harden the tool_pipedrive path and bulk-delete helpers

Two follow-ups from the second CodeRabbit pass, both on helpers added in the
previous commit.

_delete_bulk merged the caller's `extra` pass-through after assigning the
validated `ids`, so `deal_delete_bulk({'ids': [1], 'extra': {'ids': '2,3'}})`
deleted 2 and 3 instead of 1. Merge `extra` first and let the validated CSV win.
The hand-rolled version this replaced had the same ordering, so the flaw predates
the helper.

path_segment relied on quote() alone, but "." and ".." contain nothing quote
escapes — `/notes/{id}/comments/..` survived encoding intact and addressed the
parent resource. Reject those two segments instead of encoding them.

Tests cover both: a dot-segment id raises before any request is made, and a
pass-through "ids" no longer displaces the validated list while other extra
parameters still reach the query string.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes, ai): keep the Pipedrive token out of transport error text

The personal-token path puts the credential in the `api_token` query parameter,
so requests bakes it into the URL. A transport-level failure (DNS, refused
connection, TLS, timeout) raises a RequestException whose text carries that URL,
and `raise ValueError(f'Pipedrive request failed: {exc}') from exc` copied the
token verbatim into the error the agent framework surfaces and logs. The chained
cause held the same URL, so scrubbing the message alone would still have leaked
it through any printed traceback.

Build the message from the exception type plus method and path — all secret-free,
and the type still distinguishes ConnectTimeout from SSLError from ConnectionError
— and raise `from None`. Verified this was the only such site: the JSON-decode
path stringifies `resp.text[:200]` rather than the URL, `_raise_pipedrive_error`
builds from parsed response fields, and the tenacity wrapper uses `reraise=True`
so no RetryError repr is introduced.

Three tests pin it, and all three fail against the previous code: the token
appears in neither the message nor the formatted traceback (that half is what
pins `from None`), and the message still names the failure and the call.

Also:
- notes' `add_time` was in _NOTE_WRITE_KEYS but not _NOTE_WRITE_PROPS, so
  body_from could never populate it — the same class fixed for deals,
  organizations, persons and leads in 7b3cc53. A scan of every
  *_WRITE_KEYS/*_WRITE_PROPS pair confirms notes was the last one.
- ChatBase._chat had no direct test. flatten_content_blocks and extract_text are
  covered, but nothing pinned the wiring that routes the provider's content
  through them, which is the contract every non-streaming chat caller depends on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
nihalnihalani added a commit to nihalnihalani/rocketride-server that referenced this pull request Sep 4, 2026
The bridge was written against a monitor protocol that carried no event
timestamps, so every span used the bridge's own clock at event arrival and
the guide documented that as a limitation. Since rocketride-org#1661 the engine stamps
`body.eventTime` (epoch seconds, set once at ingress by the run-log
continuum) on every forwarded event, and derived flow events inherit their
source event's stamp — so the limitation is no longer true and the
documentation shipped with it was wrong against develop.

Span start/end and span-event timestamps now come from `body.eventTime`,
making durations engine-measured instead of bridge-observed: WebSocket
transport latency and event-loop scheduling no longer inflate every
component span.

The fallback is exactly the previous behaviour. A body with no usable
`eventTime` (an older engine, or a non-float/NaN/non-positive value) and
every span closed outside an event — shutdown, seeded-snapshot
reconciliation, tracked-run cap eviction — use `time.time_ns()`, the same
clock the SDK would have used. Because a run can therefore mix both
sources, each open span records its own start and a close is clamped to
never precede it, so no span can report a negative duration.

Metric points are unchanged: OpenTelemetry metric instruments take no
explicit timestamp, so they remain arrival-timed. That is now stated
rather than left implicit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
asclearuc pushed a commit that referenced this pull request Sep 15, 2026
…mented ingester protocol (zero engine changes) (#1612)

* feat(sdk): rocketride.otelbridge — OpenTelemetry export over the documented ingester protocol (#1609)

Maps the engine's monitor event stream to standard OTel telemetry with
zero engine/server changes — the client-side refinement invited by
tracking issue #616 after the engine-invasive #498 was archived.

- mapper.py: stateful FlowSpanMapper reconstructs span trees from
  apaevt_flow — task root span, pipe span per pipe id, component child
  spans paired by component identity (per the observability doc's
  warning against stack-position pairing); trace.error -> span status
  ERROR + exception event; apaevt_sse -> span events; unclosed spans at
  end/shutdown flagged rocketride.span.unclosed=true; per-pipe state is
  bounded. MetricsMapper maps apaevt_status_update snapshots to OTel
  instruments (object counts, rates, cpu/gpu memory + peaks).
- Semantic conventions per the July 2026 semantic-conventions-genai
  snapshot: gen_ai.provider.name (not the deprecated gen_ai.system),
  gen_ai.operation.name from the well-known list only, span names
  '{operation} {model}' style, INTERNAL span kind for in-process
  component executions.
- Privacy by default: lane payloads / prompt / result content never
  reach attributes or events unless include_content is set (8KB
  truncation cap when it is); enforced by a sentinel-scan test across
  all exported attributes and events.
- setup.py: TracerProvider/MeterProvider + OTLP http/protobuf exporters;
  ALL opentelemetry imports are lazy so the base package imports without
  the extra.

53 unit tests (InMemorySpanExporter/InMemoryMetricReader; importorskip
so the base CI matrix stays green), including a full replay of the
captured real-engine event fixture.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(cli): add 'rocketride otel' subcommand + optional [otel] extra (#1609)

rocketride otel [--endpoint URL] [--protocol http|grpc] [--service-name]
[--headers k=v,...] [--include-content] [--no-metrics] + the standard
connection args. Subscribes via rrext_monitor with token '*' (the
observability doc's recommended ingestion scope, types TASK|SUMMARY|
FLOW|SSE), dispatches to the mappers, reconnects with capped backoff
and resubscribes (subscriptions are per-connection and not durable),
and flushes exporters on SIGINT/SIGTERM.

Endpoint resolution honors the standard OTLP env contract: explicit
--endpoint > OTEL_EXPORTER_OTLP_TRACES_ENDPOINT (verbatim) >
OTEL_EXPORTER_OTLP_ENDPOINT (base URL + /v1/traces|/v1/metrics) >
default. Packaging: opentelemetry-sdk + OTLP http exporter live behind
an optional extra — the base install gains zero dependencies, and
running the subcommand without the extra exits 2 with the exact
"pip install 'rocketride[otel]'" hint before any connection attempt.

25 tests: config precedence matrix, dispatch/reconnect/resubscribe with
a fake client, shutdown flush ordering, missing-extra exit path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: OpenTelemetry bridge guide with Jaeger, Langfuse, and Datadog recipes

Setup guide covering the span model (task -> pipe -> component tree),
the privacy default and --include-content, the FLOW-requires-trace-level
callout, reconnect semantics, and copy-paste recipes with exact OTLP
endpoints/auth for Jaeger all-in-one, Langfuse (/api/public/otel, Basic
auth), and Datadog. Notes that Jaeger does not ingest OTLP metrics —
pair it with --no-metrics or point metrics at a collector.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(sdk): handle partial OpenTelemetry installs — guard the HTTP exporter import

CI exposed a real environment our matrix missed: opentelemetry-api/sdk
present as another package's transitive dependency (tool_daytona pulls
them in) while the OTLP HTTP exporter is absent. Two fixes:

- setup.py: the http trace/metric exporter imports are now guarded like
  the grpc ones — a partial install raises OtelNotInstalledError with
  the pip install 'rocketride[otel]' hint (CLI exit 2) instead of a raw
  ModuleNotFoundError. Verified live in a reproduced partial venv.
- test_otel_setup.py: importorskip targets the most specific module the
  tests exercise (opentelemetry.exporter.otlp.proto.http), so the module
  skips cleanly in CI's partial environment; plus a regression test that
  simulates the partial install via a None sys.modules entry and asserts
  the friendly error.

Partial venv: 63 passed + 1 module skip; full extra: 79 passed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(sdk): OTel bridge review round — privacy-gate errors, run-id promotion, header precedence, shutdown ordering

CodeRabbit review round on #1612, each finding verified before acting:

- mapper: trace.error no longer bypasses the privacy gate — by default
  only the error signal exports (status ERROR, bare exception event,
  generic description); --include-content restores verbatim text, now
  subject to the same 8KB truncation strings previously skipped. The
  sentinel privacy test plants the sentinel in trace.error.
- mapper: a run first seen via flow events (before the task 'running'
  snapshot) is now promoted from its project|source fallback key to the
  canonical run id when it arrives — previously a second run state was
  created (broken parentage) and the orphaned fallback state leaked
  permanently. Confirmed with a failing repro; guard test ensures two
  distinct canonical runs sharing (project, source) never merge.
- config/setup: only --headers become explicit exporter headers; env
  headers are left to the SDK so OTEL_EXPORTER_OTLP_TRACES_HEADERS /
  _METRICS_HEADERS signal precedence works (it was silently defeated).
  Precedence matrix tests added.
- setup: meter provider shutdown wrapped in finally so metrics flush
  even when tracer shutdown raises.
- CLI: --trace-level none message now says flow tracing stays disabled;
  docs recipes pass collector auth via OTEL_EXPORTER_OTLP_HEADERS env
  instead of argv (shell-history hygiene); error-mapping doc wording
  matches the implementation (component leave events).

Full extra: 90 passed. Partial install (api/sdk without exporter):
69 passed + clean skip. ruff clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(sdk): absorb stranded fallback run state when canonical state precedes it

Follow-on to the run-promotion fix: an SSE event creates the canonical
run state WITHOUT registering the (project, source) alias (SSE bodies
carry no project/source per the protocol), so an id-less flow for the
same run could still open a parallel fallback-keyed state — and because
the canonical state already existed, the promotion branch was skipped,
stranding the fallback state (kept alive forever by snapshot
reconciliation) with duplicate spans.

_get_run now reconciles whenever an event carries both the canonical id
and project/source: no canonical state → promote (unchanged); canonical
state exists → _absorb_fallback_run migrates non-colliding pipes (open
spans re-stamped with the canonical run id), closes colliding pipes as
rocketride.span.unclosed=true, and drops the fallback key. Only
fallback-format keys are touched, so distinct canonical runs never
merge (guard test unchanged and green).

Both new regression tests failed before the fix. Full extra: 92 passed;
partial install: 71 passed + expected skip; ruff clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(sdk): stamp OTel spans with the engine's body.eventTime

The bridge was written against a monitor protocol that carried no event
timestamps, so every span used the bridge's own clock at event arrival and
the guide documented that as a limitation. Since #1661 the engine stamps
`body.eventTime` (epoch seconds, set once at ingress by the run-log
continuum) on every forwarded event, and derived flow events inherit their
source event's stamp — so the limitation is no longer true and the
documentation shipped with it was wrong against develop.

Span start/end and span-event timestamps now come from `body.eventTime`,
making durations engine-measured instead of bridge-observed: WebSocket
transport latency and event-loop scheduling no longer inflate every
component span.

The fallback is exactly the previous behaviour. A body with no usable
`eventTime` (an older engine, or a non-float/NaN/non-positive value) and
every span closed outside an event — shutdown, seeded-snapshot
reconciliation, tracked-run cap eviction — use `time.time_ns()`, the same
clock the SDK would have used. Because a run can therefore mix both
sources, each open span records its own start and a close is clamped to
never precede it, so no span can report a negative duration.

Metric points are unchanged: OpenTelemetry metric instruments take no
explicit timestamp, so they remain arrival-timed. That is now stated
rather than left implicit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(client-python): publish the OTel bridge guide and mirror its CLI entry

Two co-location gaps (AGENTS.md:5-28) and one stale claim.

The 369-line guide lived at docs/README-otel-bridge.md, and root docs/ is
not in the docs-site glob (packages/docs/scripts/lib/gather.js:25), so the
guide was GitHub-only. Moved to packages/client-python/docs/otel-bridge.md,
the tree already mounted at develop/python (packages/client-python/scripts/
tasks.js:236) alongside analytics.md and log.md, so it is gathered to the
site at /develop/python/otel-bridge. Its in-repo links become site routes
to match the sibling pages; the three references to it were repointed.

packages/client-python/docs/index.md is the site's Python page and mirrors
the PyPI README's ## CLI block line for line, but had no `rocketride otel`
row and none of the OpenTelemetry paragraph the README gained. Added both.

The guide's ## Limitations still said span timestamps are arrival time
because the wire carries none; the previous commit made spans engine-timed,
so that bullet now describes what actually remains bridge-timed (spans
closed outside an event, pre-continuum engines, and all metric points).

The Python blocks in the guide are also reformatted: ruff formats Markdown
code blocks as of 0.16, and CI's ruff-action is unpinned, so the file failed
`ruff format --check` today at either path. Repo-wide check is clean again.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(changelog): record the rocketride otel bridge under [Unreleased]

CHANGELOG.md [Unreleased] ### Added is the convention for user-visible
additions and is kept alphabetical by scope; a new CLI command and a new
optional extra qualify.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(client-python): name Grafana Tempo in the OTel backend recipes

Issue #1609's acceptance list says "Jaeger/Tempo", and the guide covered
Tempo only implicitly as "Grafana stacks". Tempo is the OTLP receiver a
Grafana user actually points the bridge at, and it needs one non-obvious
step (enable distributor.receivers.otlp) plus a separate metrics backend,
so it is worth naming rather than leaving a reader to infer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(sdk): address CodeRabbit review round on the rebased head

Nine threads on the rebased head, all in the OTel bridge. Dispositions:

- docs/otel-bridge.md:151 (Sensitive Data Exposure, CWE-522) — FIXED. The
  Langfuse and LangSmith recipes now prompt for the keys (printf + read -rs,
  spelled identically in bash and zsh) instead of embedding placeholders that
  operators replace with live secrets in a command line, and point at a secret
  manager as the better source. The base64 pipeline is now
  `printf '%s' ... | base64 | tr -d '\r\n'`: real Langfuse keys make the
  joined string 85 characters, past GNU base64's 76-character wrap, and
  command substitution strips only trailing newlines, so the embedded break
  would have produced an invalid Authorization header.

- cli/commands/otel.py:207 (Sensitive Data Exposure, CWE-532) — FIXED. The
  startup line ran `config.endpoint` straight to stdout; an endpoint can carry
  userinfo credentials or a signed query string. It is now always passed
  through `redact_endpoint()`, which drops userinfo, query and fragment while
  keeping scheme/host/port and the ingest path operators actually verify.

- cli/main.py:466, otelbridge/config.py:177, otelbridge/setup.py:123
  (Cleartext Transmission, CWE-319) — FIXED, once, in one place.
  `validate_transport_security()` runs before any exporter exists (in
  `build_providers`, and again in the CLI so the failure is exit code 2 rather
  than a traceback): a credential-looking header name from `--headers` or any
  of the three OTEL_EXPORTER_OTLP_*HEADERS variables, plus a non-loopback
  cleartext endpoint for either signal, aborts startup. Header VALUES are
  never read or echoed — only names reach the message. Loopback stays exempt
  for local collectors; `--insecure` / ROCKETRIDE_OTEL_ALLOW_INSECURE=1 is the
  explicit opt-out and warns on stderr.

- otelbridge/config.py:189 (CWE-201, redirect header forwarding) — FIXED. The
  OTLP/HTTP exporters now get a session that never follows redirects.
  `requests` strips Authorization only when the redirect changes host, so a
  3xx replayed x-api-key to the target and even kept Authorization on an
  https -> http downgrade to the same host.

- otelbridge/config.py:177 (endpoint precedence) — FIXED. `from_args_env` no
  longer pre-reads OTEL_EXPORTER_OTLP_ENDPOINT into an explicit exporter
  endpoint: doing so overrode the signal-specific _TRACES_ENDPOINT /
  _METRICS_ENDPOINT, inverting the OTLP spec order — the same defect already
  fixed for the header variables in c23b312. The SDK now resolves all three,
  and `effective_endpoint()` mirrors its order read-only for the startup line
  and the security check.

- otelbridge/bridge.py:207 (signal ownership) — FIXED. `loop.add_signal_handler`
  replaces rather than chains, and asyncio exposes no way to restore the
  displaced callback, so the bridge no longer touches a signal the running
  loop already handles; `install_signal_handlers=False` turns registration off
  entirely for embedders.

- otelbridge/mapper.py:611 (unbounded run.pipes) — FIXED. MAX_TRACKED_RUNS did
  not bound pipes: a run that keeps receiving events is re-inserted as
  most-recently-eventful and never evicted, while every begin (or event for a
  never-begun pipe) adds a _PipeState that only a matching end removes, and a
  reconnect does not replay missed ends. MAX_TRACKED_PIPES_PER_RUN now closes
  the least-recently-eventful pipe as unclosed, with recency updated on pipe
  activity rather than inferred from insertion order.

- otelbridge/setup.py:202 (tracer leak) — FIXED. A failure anywhere in the
  metric half left the BatchSpanProcessor worker running with no handle to
  stop it, because `shutdown` had not been created yet. That path now shuts
  the tracer provider down before re-raising.

Tests: 32 added (131 in tests/test_otel_*.py, was 99), covering stdout
redaction, the refusal and its opt-out, env-var and signal-specific endpoint
coverage, the non-redirecting session, the tracer teardown, signal ownership
and the pipe cap's LRU behaviour.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(sdk): skip the loop signal-handler ownership test on Windows

asyncio's ProactorEventLoop raises NotImplementedError from
add_signal_handler, so the test cannot set up its precondition there.
The bridge itself already tolerates that platform (the other signal
tests pass on Windows); only the test needs the platform guard.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(cli): document the otel --insecure flag in the site CLI reference

The `otel` command grew an `--insecure` opt-out for the cleartext-credential
guard, but the docs-site "Key flags" table was never updated to match. Users
hitting exit code 2 from `rocketride otel --endpoint http://tempo:4318
--headers Authorization=...` had no way to learn why from the CLI reference —
the flag was only documented in the client-python OTel bridge guide.

Add the missing row, naming the `ROCKETRIDE_OTEL_ALLOW_INSECURE` env var
alongside the flag so the site table matches otel-bridge.md and the argparse
help text.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(sdk): declare the otel test dependency and stop orphaning bridge providers

Two review findings on the `rocketride otel` bridge.

1. The opentelemetry dependency of the bridge tests was inherited, not
   declared. tests/test_otel_mapper.py and tests/test_otel_setup.py opened
   with pytest.importorskip('opentelemetry'), and nothing in the test path
   installs the 'otel' extra — the packages reached the engine interpreter
   only as a transitive dependency of the crewai node. Dropping or repinning
   that node would have turned ~1450 lines of mapper/setup coverage into a
   silent skip with CI still green.

   CI never pip-installs the package (pytest runs on the engine interpreter
   importing the package from the source checkout via tests/conftest.py), so the extras alone cannot fix this. The
   deps are added to packages/server/build-requirements.txt, which
   server:setup-test-deps installs through depends() under the merged
   constraints — the one install step that path goes through — and the
   'dev'/'test' extras now pull 'rocketride[otel]' so a source checkout
   matches. Both importorskip guards become hard imports: a missing
   dependency now fails collection instead of skipping.

2. build_providers() always built both provider halves and adopted its
   shutdown only when shutdown_fn was None, so an embedder passing its own
   shutdown_fn left a TracerProvider and its BatchSpanProcessor worker
   thread with nothing to stop them — and the half the caller had already
   supplied via a factory was built, wired and discarded.

   build_providers() takes with_tracer / with_meter (defaulting to the old
   behaviour) and run_bridge asks only for the half it is missing; a
   caller-supplied shutdown_fn is now chained in front of the providers'
   shutdown — caller first, providers in a finally — instead of replacing
   it. The CLI path passes neither argument and is unaffected.

Regression tests cover chain order, the provider shutdown surviving a
raising caller, and which half is built for each factory combination;
test_otel_bridge.py stays runnable without the extra. The embedded-use
ownership rules are documented in the bridge doc.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* build(client-python): install the otel test deps in the test step, not the engine

Review follow-up: the previous commit declared opentelemetry-sdk and
opentelemetry-exporter-otlp-proto-http in packages/server/build-requirements.txt.
That file is installed by server:setup-test-deps on every engine build, so every
engine install carried the OTel SDK and the OTLP exporter whether or not anyone
ran the bridge — which made the summary's "zero new base dependencies" true of
the package but not of the engine.

Move the cost to the step that needs it. A new client-python:setup-test-deps
action installs packages/client-python/tests/requirements.txt into the engine
interpreter through depends(), and runs only inside client-python:test, between
the parallel build group and the test-server bracket:

  server:build -> [nodes:build | ai:build | client-python:build]
               -> client-python:setup-test-deps
               -> py-test-server:setup -> client-python:run-pytest -> teardown

That ordering puts it after the node/ai requirement files land in the dist, so
depends() resolves against the complete constraint set, and before the bracket,
so a cold install does not hold the test server open. This mirrors the existing
packages/ai/tests/requirements.txt pattern (packages/ai/scripts/tasks.js).

packages/server/build-requirements.txt is restored byte-for-byte to its previous
state. The 'rocketride[otel]' self-reference in the dev/test extras stays — that
is the source-checkout path — and the hard imports in test_otel_mapper.py /
test_otel_setup.py stay, so a missing dependency still fails collection instead
of skipping. The tests no longer depend on the crewai node pulling opentelemetry
in transitively either way.

Docstrings in both test modules and the extras comment in pyproject.toml now
point at the new file and step.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(client-python): surface the OTel bridge on the Python SDK overview

Every other page under docs/public/python/ has a card in the SDK overview's
"Explore the SDK" grid; the newly re-homed otel-bridge page did not, so it
was reachable only from the sidebar and the CLI reference.

Adds the card last in the grid, matching sidebar_position: 13. Uses LuRadar,
which is not already taken by another card in this file and is present in the
Lucide set react-icons v5 ships.

This replaces the "OpenTelemetry export (`rocketride otel`)" section that used
to live in docs/README-python-client.md and packages/client-python/docs/index.md
— both deleted by the docs revamp (#2139).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(clients): declare otel-bridge as a Python-only site page

The client-docs gate requires both folders to publish the same page set, with
single-language extras declared in two places: PY_ONLY_PAGES/TS_ONLY_PAGES in
scripts/validate-client-docs.py, and the Site page set section of the schema.
docs/public/python/otel-bridge.md was added without either, so 'Docs site build'
failed with: python: undeclared site page otel-bridge.md.

The page is genuinely Python-only — the TypeScript SDK ships no OTel bridge
(no otel surface under packages/client-typescript/src, no page in
docs/public/typescript/) — so it belongs in PY_ONLY_PAGES rather than being
mirrored. Declared at sidebar_position 13, matching the TypeScript-only
database-sequelize extra.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(sdk): name the private SDK attribute the OTel bridge hooks

The bridge receives monitor events by assigning its dispatcher to
RocketRideClient._caller_on_event — the attribute EventMixin.__init__
creates from the public on_event= kwarg and EventMixin.on_event awaits
for every DAP envelope. It assigns rather than passing on_event= because
it is handed an already-built client and must restore the previous
handler on exit.

Nothing in the SDK's public surface pins that name, and a rename would
fail silently: assigning an unknown attribute to a Python object
succeeds, so the bridge would keep connecting, keep subscribing, keep
logging "reconnected", and never export another span.

Lift the string into CLIENT_EVENT_HOOK with the rationale next to it, so
the coupling is a single named seam a test can assert on instead of a
literal repeated across three call sites. No behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(sdk): pin the OTel bridge against the real RocketRideClient

The otel suite was mock-only past the mapper: test_otel_bridge.py and
test_otel_cli.py each define their own FakeClient, and the provider tests
monkeypatch build_providers, FlowSpanMapper and MetricsMapper out. That
covers the run loop's branches but leaves the bridge's one point of
coupling to the SDK untested — and a fake that defines
_caller_on_event itself can never notice the SDK dropping it.

Measured: renaming _caller_on_event to _caller_on_event_renamed across
mixins/events.py and mixins/data.py — a change that stops the bridge
exporting anything at all — leaves all 141 existing otel tests green.

Add tests/test_otel_client_contract.py, which constructs a REAL
RocketRideClient and stubs only the transport (call/is_connected); the
SDK's add_monitor bookkeeping, EventMixin.on_event dispatch, the mappers
and the OpenTelemetry provider pipeline all run for real:

  - three tests pinning CLIENT_EVENT_HOOK from different angles: the
    attribute exists in vars() on a real client, it is where the public
    on_event= kwarg lands, and the SDK's dispatch actually awaits it
  - run_bridge end to end on a real client, with events pushed through
    EventMixin.on_event rather than a fake's emit(), asserting routing,
    the unconsumed-event case, and hook restoration on shutdown
  - the wildcard subscription as the real SDK encodes it on the wire,
    plus the _monitor_keys entry the bridge relies on for replay after a
    reconnect
  - the 24-record wire fixture replayed to real exported spans, matching
    the 10-span forest test_otel_mapper.py gets by feeding the mappers
    directly, so the bridge provably adds, drops and reorders nothing

Under the same simulated rename, six of these seven fail.

Also add an opt-in live export against a real OTLP collector, gated on
ROCKETRIDE_OTEL_TEST_ENDPOINT via the same skipif shape
RocketRideClient_test.py uses for requires_llm. It starts no collector
and changes no CI step; without the env var it simply skips.

Note the new file in the client-python:setup-test-deps comment (it
imports the OTel SDK at collection time) and cross-reference it from
test_otel_bridge.py's docstring.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: nihalnihalani <24360630+nihalnihalani@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
AviNormie pushed a commit to AviNormie/rocketride-server that referenced this pull request Nov 9, 2026
…e identity (rocketride-org#1661)

* feat(task): L1 run-log source stamping — header eventTime + continuum seq on every task event

Foundation phase of the run-logging plan (per-task JSONL event continuum).
Every task event now carries two header-level fields, stamped server-side
exactly once:

- eventTime (epoch seconds, float): stamped at the stdio ingress where raw
  engine stdout frames become JSON events (TaskDbgStdio.on_event), so it
  reflects emission time, not forwarding time. Derived events (apaevt_flow)
  inherit their source trace event's time. An engine-provided stamp is never
  overwritten, leaving the door open to in-engine stamping later.

- seq: a per-task continuum sequence seeded from the epoch in MICROSECONDS.
  Why epoch-us: a new process's seed always exceeds every seq a previous run
  of the same task identity issued (nothing sustains 1M events/sec), which
  makes seq monotonic across runs, engine restarts, and supervisor crashes
  with no persisted counter handshake; ~1.78e15 stays under JS
  Number.MAX_SAFE_INTEGER. Legacy small per-endpoint DAP counters (from
  build_event) are re-stamped; values at/above CONST_LOG_SEQ_FLOOR (1e14)
  are recognized as continuum seqs and left untouched (idempotent).
  Request/response seq is never rewritten — DAP correlates responses via
  request_seq. raise_log_seq_floor() lets the L2 run-log writer lift the
  seed to max(seed, control.lastSeq + 1) as belt-and-suspenders.

_forward_task_event stamps idempotently as a safety net so every broadcast
path (status updates, SSE, output) is covered by the same single counter.

Transport: plain output lines now preserve their originating channel as the
output category ('stderr' vs 'console') so the run log and Log page can
distinguish engine diagnostics from console prints.

Client folds (shared-ui project module): TraceEvent carries the new required
headers via parseServerEvent, and useTraceState row timestamps are now
event-sourced (eventTime * 1000). Date.now() is ELIMINATED from the fold —
no wall-clock fallback — so the same fold produces identical rows live and
on run-log replay (the L3+ player/DVR depend on this).

New logging constants (segment size, ring, chapters cap, payload caps,
history ages, sampling, read paging) land together in ai/constants.py and
are consumed progressively by later phases.

Part of the run-logging implementation plan (L1 of L1–L5).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(task): L2 run-log writer — per-task JSONL continuum with sealed segments, control file, chapters, retention

One continuous log per task identity (projectId.source.runKind): runs are
chapters (lifecycle marker events) INSIDE the stream, never separate files —
which also resolves the deterministic-token collision (successive runs of
the same pipeline reuse the same identity, so per-run files were ambiguous
by construction).

RunLogWriter (new ai/modules/task/run_log.py):
- Append path: filtered event types (output, flow, errors, exit, lifecycle)
  + sampled status snapshots (one per CONST_LOG_STATUS_SAMPLE_SECONDS) +
  per-event payload truncation that always preserves metadata/timestamps.
  Synchronous and store-free — all store traffic is in a background worker.
- Sealing: 16 MB size seal on a line boundary (JSONL segments concatenate
  byte-for-byte) plus a 24 h backstop seal so low-volume long-lived streams
  cap host-loss tail and store lag at <= 1 object/day. Every store object is
  written exactly once and never modified — no compaction, no multipart
  visibility concerns, identical behavior on fs/S3/Azure/memory providers.
- Control file (./logs/{projectId}.{source}.{runKind}.json under the user's
  store scope; token-free paths AND token-free log content): the time/seq ->
  segment routing table and spooled/uploaded location ledger (segments carry
  startTime/endTime/chapterStart/seq/id/state), stream startTime/endTime,
  lastSeq, and the capped chapters list that renders the UI activity bar
  from one small read.
- Retention: 64-segment ring + per-kind history age (dev 7d / deploy 30d);
  eviction deletes store object AND spool copy (lease-deferred) and advances
  the horizon, trimming chapters with it. Long-lived streams never finalize,
  so the ring is their steady state.
- Concurrency disciplines: state flips to 'uploaded' BEFORE any spool
  delete; SegmentLeases defers (never skips) deletes under readers; upload
  failure leaves the spool copy serving reads and retries later.
- Recovery is STORE-SIDE ONLY (K8s container replacement kills the spool):
  reopen trusts 'uploaded' entries, drops 'spooled' ones (the accepted loss
  window), lifts the task's seq floor to lastSeq + 1 over the epoch-us seed
  (belt and suspenders), and records a clock-anomaly lifecycle event in the
  stream itself if the wall clock ever sits below the persisted lastSeq.
- Startup hygiene: sweep_spool_root() deletes stale spool dirs at module
  init — spool state is never salvaged, and dev machines (whose filesystem,
  unlike a replaced container, survives restarts) cannot leak disk.

Task wiring (task_engine.py): writer opens at subprocess start (run-begin
marker doubles as the chapter header; pipeline config hash, trigger, trace
level — never tokens), appends at the _forward_task_event choke point (what
clients see is exactly what replay reproduces), completes the chapter with
its outcome at termination, and records restarts as markers WITHOUT opening
a new chapter (a restart is not a new run). Logging is best-effort: any
failure leaves the task running unlogged.

Tests: 21 new unit tests over the filesystem provider (sealing, backstop,
upload transitions incl. failure/retry, two-run continuum + chapters, ring
and age eviction from both locations, chapters cap, store-side recovery,
clock-anomaly fallback, truncation, leases, spool sweep) + a conftest that
stubs the dist-only bootstrap modules (depends/rocketlib) so the suite runs
against the source tree; real modules always win when present. Full ai
suite: 1389 passed. test_task_engine's bare-Task helper seeds the two new
attributes _forward_task_event now consults.

Part of the run-logging implementation plan (L2 of L1–L5).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(task,client): L3 rrext_log — one read path over the run-log continuum + SDK log namespaces

Server:
- RunLogReader (run_log.py): ranged/paged reads routed via the control file
  (seq/time/segment range forms), each segment resolved SPOOL-first with
  store fallback under read leases held only for the file read (a lazy
  generator would leave lease release to GC timing when a page limit stops
  mid-segment — deterministic release instead). Horizon-honest responses
  (truncatedAtSeq) and cursor paging (nextSeq).
- The ACTIVE (unsealed) segment is now visible to readers: seal-or-continue
  keeps a completed run's tail (incl. its run-end marker) in the active
  spool file, so the control file carries an 'active' descriptor and the
  reader includes it as a virtual segment — the tail is readable the moment
  it is written. end_run closes the file HANDLE (no fd leak) but keeps the
  file + descriptor; the next run in-process reopens and continues it,
  restoring the descriptor so the shared segment keeps one coherent
  start time/seq.
- Recovery unified under one rule — trust the filesystem: 'uploaded'
  entries always kept; 'spooled'/'active' entries kept (uploads re-queued)
  when their spool file exists (same-process reopen) and dropped when it
  died with a previous container (the accepted loss window).
- WRITERS registry: a live stream's reader prefers the writer's in-memory
  control (fresher than the store copy by up to a seal interval), and
  delete routes through the writer's lock so it never races seal/upload.
- delete: beforeTime (segment-granular — drop segments wholly older; trim
  chapters; advance horizon) or all (remove segments + control), both
  locations, lease/ordering disciplines.
- cmd_log.py: LogCommands TaskConn mixin (subcommand table: chapters /
  read / delete) in the cmd_store/cmd_deploy pattern. Streams addressed by
  the plain identity tuple (projectId + source + runKind) — NEVER by token;
  store scoping derives from the AUTHENTICATED user only. task.monitor for
  reads, task.control for delete.

SDKs (both, in sync): client.log namespace — chapters()/read()/delete() —
with full type surfaces (LogChapter, LogActivitySpan, LogChaptersResult,
LogEvent, LogReadResult, LogDeleteResult, LogRunKind) in TS and Python,
plus the co-located guide page (docs/guide/methods/log.md) documenting the
range forms, paging protocol, stamped headers, and permissions.

Tests: 11 reader tests (chapters shape, ranged reads incl. types filter and
time ranges, cursor paging terminates + matches unpaged, horizon
truncation, spool-gone store fallback, delete beforeTime segment
granularity + delete all, live-writer control preference). Full ai suite:
1400 passed.

Part of the run-logging implementation plan (L3 server/SDK of L1–L5).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(shared-ui,task): L3–L5 delivery layer — useTaskEvents smart buffer, PlayBar DVD transport, Log + Analyze projections

useTaskEvents (modules/project/hooks) — THE single event-delivery point for
one source's continuum, with media-player semantics per the plan:
(1) DVR ring buffer: live events accumulate (capped, oldest dropped) even
while paused or scrubbing — pause-live/scrub-back/GO LIVE need no refetch;
(2) speed-aware prefetch: replay pages ahead of the playhead via the
host-injected log.read pager, runway scaled by speed; (3) generation-
cancelled seeks: in-flight fetches from a previous position are discarded
on arrival; v1 contiguous-window policy (near seek reuses the buffer, far
seek flushes + refills); (4) live/backfill merge deduped on seq; (5) pacing
decoupled from fetching — dispatch at eventTime deltas / speed from the
buffer. Also detects idle gaps between tracks and exposes the DVD
controller (prev/next track, +/-30s skip, speeds, go-live); PLAY from a gap
auto-skips to the next track's begin. Host-agnostic: readLog is injected,
live events arrive via ingestLive — one instance per source section.

PlayBar (components/play-bar) — the per-source transport + activity bar:
|prev / rewind-30s / play-pause / forward-30s / next| + 1X/2X/4X/10X/25X +
GO LIVE, over the single-lane timeline rendered PURELY from log.chapters
metadata (no segment reads until a seek): filled spans per track (error
tracks red, live span green, minimum span width keeps 3-second runs
clickable), click-to-seek, cursor knob, retention-horizon axis label.

LogPane — the human-readable projection of output/stderr/lifecycle events
(the "Log page is a projection, not a second file" decision), with
tail-pinning that detaches on manual scroll. AnalyzePane — efficiency
analysis over the visible window from server-stamped emission times:
per-component latency (reentrancy-safe enter/leave pairing), per-document
durations, and idle gaps between runs — the trace-analyze concept, now on a
real data substrate.

run_log.py: the active segment opens line-buffered, so every appended event
reaches the OS immediately and rrext_log reads of the ACTIVE segment are
current to the last event — gap-free live composition (store + spool +
active) without a separate in-memory tail structure (the L5 read-v2
mechanism, delivered by construction).

Part of the run-logging implementation plan (L3 UI delivery + L4 player +
L5 pieces of L1–L5).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(task,dap): preserve continuum stamps across the monitor event forward

The monitor forward (MonitorCommands.send_task_event) rebuilt each outgoing
envelope with build_event(), which mints a fresh per-connection DAP seq and
carries no eventTime — so live subscribers never saw the continuum stamps
the run log records, even though every task event is stamped at stdio
ingress. That silently broke everything that correlates live events with
the logged stream: the client replay folds (which have no wall-clock
fallback by design), the live/backfill dedupe seam in useTaskEvents, and
the late-joiner DVR protocol (first live seq X -> read toSeq X-1).

send_event() now takes optional event_time/seq kwargs that restore the
original header stamps onto the built envelope (only as a pair, matching
how Task.stamp_log_event issues them), and the monitor forward passes the
stamps through from the source message. Request/response seq is untouched
— only forwarded event messages are re-stamped, per the DAP request_seq
correlation constraint. Extending send_event (rather than bypassing it to
self.send) keeps one envelope-building path and the existing test seam.

Part of the run-logging plan (L1 stamping, delivery to live monitors).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(shared-ui,vscode): source-major environment pages — SourceSection unit + Design|Development|Deploy strip

Restructures ProjectView to the final UI direction (v5, source-major),
replacing the five page-level monitoring views (Status/Tokens/Flow/Trace/
Errors) that could only ever show the live task:

- New SourceSection: the self-contained per-source unit. Each section
  carries its OWN pill bar (Status/Tokens/Flow/Trace/Errors/Log/Analyze),
  pane, and PlayBar bound to that source's run-log continuum. Live mode
  renders from host-fed state exactly as before; replay mode derives
  status from the last absolute snapshot in the visible window, folds
  trace/flow through the SAME parseServerEvent/useTraceState path used
  live, and projects Log/Analyze straight off the window. Seeking between
  tracks shows an explicit idle-gap state with previous/next-run actions
  (DVD semantics — PLAY from a gap auto-skips to the next track).

- ProjectView page strip becomes DESIGN | DEVELOPMENT | DEPLOY.
  Development stacks dev-continuum sections; Deploy hosts an optional
  pipeline-level deployLifecycle slot (owned by the deploy feature) above
  deploy-continuum sections. Sources replay independently — no shared
  cursor. Save now lives ONLY on the Design page (inside the canvas);
  the monitoring pages carry no document actions. Persisted view modes
  from before the restructure migrate onto the new strip (old monitoring
  modes -> development). The Development entry carries the live
  error/warning count so problems stay glanceable from any page.

- New host props: liveLogEvents (raw stamped envelopes; sections split
  them per source by body.source or the '<tokenhash8>.<source>' envelope
  id), readLog + fetchTimeline (stream-bound wrappers over client.log),
  and the deployLifecycle slot. traceEvents/onTraceClear are deprecated
  (kept so existing hosts compile) — sections fold trace state
  themselves, which is what makes replay identical to live.

- VS Code webview host wired live-only for v1: accumulates this
  project's stamped events (capped, chunky trim — sections detect the
  shrink and re-feed their seq-deduped buffers) and passes them through;
  no replay bindings yet. The old trace accumulator and trace:clear
  plumbing are gone.

Why source-major: each source is a separate stream/task, and coupling
their players behind one page cursor (v3/v4 designs) made mixed states —
one source live, one replaying — incoherent. One section per source is
also exactly today's multi-status-card mental model, extended.

Part of the run-logging plan (L3-L5 UI delivery).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(shell-ui,shell-api): recreate shell-api v0 with the SDK client frozen in-bundle, append-only

Adding the SDK's new client.log namespace broke the shell-api contract:
the frozen v0 bundle inlined a structural copy of RocketRideClient, and
WorkspaceProvider's `client` prop put that copy in an input (contravariant)
position — so ANY additive SDK member read as a breaking change, and
shell:freeze could not mint a successor because its own pre-check compiles
the same failing floor. Design-owner decision (Rod, 2026-07-21): the client
is an MF shared singleton the shell serves to every app, so its API surface
IS part of this contract — keep it inlined in the frozen bundle and make it
append-only, and fix the root cause instead of exempting members.

- WorkspaceProvider no longer takes client/isConnected props: it reads
  both from useShellConnection() (provider-less, ConnectionManager
  singleton — the exact source Shell was reading one level above and
  threading down). Its props are the named, documented
  IWorkspaceProviderProps. With this, the client appears in NO input
  position anywhere on the surface, which is what makes strict value
  floors compatible with additive SDK growth. Rule recorded in
  contract-hold.ts and freeze-shell-api.js: components source the client
  from useShellConnection()/getClient(), never accept it as a prop.

- api.ts exports RocketRideClient as a named type. parseExportedTypes in
  the freeze script now also collects `export declare class` names
  (previously only interface/type — exported classes silently got no
  floor), so the client receives its own covariant per-version type floor:
  Frozen = Current — SDK additions pass, removing or narrowing anything
  ever frozen fails shell-ui's own tsc.

- versions/v0.d.ts recreated (deliberate, review-visible deletion +
  fresh freeze): the new snapshot contains the full client including the
  log namespace, and the slimmed provider props. contract-check /
  index.ts / latest.ts regenerated; the floors remain their plain strict
  form for every member — no exemptions.

Known limits, accepted for now (Rod): npm-release alignment still needs
freeze-at-publish discipline; client member additions do not by themselves
mint a new frozen version (name-set detector); method-parameter bivariance
can pass some parameter narrowings.

Verified: ./builder shell:check green, ./builder shell:freeze idempotent,
full ./builder saas:build green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(run-logging): unified DVR, one-seq stamping, tool tracing, log-everything, chapter self-heal

Large live-e2e iteration batch on the run-logging continuum. Every change
below came out of driving the feature against real runs; each area lists
what changed and why the previous behavior was wrong.

ENGINE / SUPERVISOR (packages/ai)

- Seq assignment happens exactly once per DELIVERED event. Task.build_event
  now stamps the continuum headers itself (the base class's small
  per-endpoint counter is replaced at build time), stdout ingress stamps
  eventTime ONLY, and the derived apaevt_flow is built inside the
  trace-level gate. Why: ingress used to burn a seq on messages that are
  consumed without delivery (apaevt_trace becomes a derived flow event), so
  every visible flow event consumed two seqs and the continuum was riddled
  with gaps.

- Final zeroed status: _terminated's terminal broadcast IS the final status
  (gauges zeroed, body.final=true), emitted after metrics stop and the
  terminal-state assignment, with the run-log closed after it. Why: an
  earlier final-zero was overwritten by a later teardown broadcast, leaving
  charts holding a stale residual reading forever; the stream's last word
  must be the zero. Outcome check reordered (CANCELLED before exitCode==0)
  so user-stops log as 'cancelled', not 'ok'.

- Tool-call tracing: DataConn._tool wraps py_instance.invoke in
  BEGIN/ENTER/LEAVE/END >DBG events via monitorOther('DBG', ...) — the same
  monitor channel the C++ debugger uses — on the REAL pipe id the call runs
  on (caller's open pipe or the borrowed pool pipe; no synthetic ids).
  LEAVE carries result + durationMs + capped output, errors and
  PreventDefault probes included. total_pipes is emitted as 0 and the
  supervisor keeps its current count for such events. Why: control-plane
  tool calls bypassed the pipeflow entirely and were invisible in
  Trace/Flow/log/replay; the recorded durations immediately exposed a
  21s-per-connection network stall that had been misattributed to the
  transport.

- run_log records EVERY event type delivered to clients; the
  LOGGED_EVENT_TYPES whitelist is deleted. Why: it is a LOG — replay and
  post-hoc analysis need what clients actually saw, and a whitelist
  silently drops any newly added event type. Status sampling (final-zero
  exempt) and the per-event payload cap still bound volume.

- Dangling-chapter self-heal: opening a stream closes any chapter left with
  endTime=null as outcome 'interrupted', ended at the stream's last
  recorded activity (capped at the new run's begin). Why: killed/crashed
  runs never write run-end; consumers treated every dangling chapter as
  live (multiple LIVE entries in the run menu) and merged consecutive runs
  into one track.

- Storage via the account FileStore (refuses an empty client_id) with
  .logs/{projectId}/{source}.{runKind}.{seg:06d}.jsonl segments + per-stream
  control file; flat $TEMP spool with pattern-anchored sweep; end_run
  force-seals and uploads so the store holds the full run immediately.
  Monitor forward passes eventTime/seq through so clients see the same
  stamps the log recorded. Tests rewritten for the FileStore layout and the
  record-everything append.

CLIENT DATA MODEL (shared-ui useTaskEvents)

- One DVR: a single seq-ordered buffer and a single position; 'live' is the
  position pinned to the wall clock, replay is the same position moving at
  speed; one transport ticker owns the clock. Why: live and replay were
  separate code paths that diverged (frozen charts, jumped positions,
  trailing dead space skipped on catch-up).

- Shaped reads: statusAt (last snapshot at-or-before position),
  trackEvents (effective track begin -> min(position, end)), chartSeries
  (1Hz grid merging tracks and dead zones), rangeEvents (Analyze brush),
  and trackStats (Now/Avg/Peak/Min scoped to the effective track). Why for
  trackStats: the chart is fixed at a one-minute window, so window-scoped
  stats read zero whenever the run's activity fell outside the visible
  minute; track scope answers what the run actually did.

- Late-joiner backfill: entering live with a log reader fetches the whole
  open track (bounded fallback window until the chapters cache reveals the
  begin, extending backward when it does), deduped on seq, draining
  multi-page continuations on the pinned tick. Why: a view opened mid-run
  had nothing from before its subscribe instant — status kept the chart
  alive but Log/Trace/Flow stayed empty forever.

- Track resolution anchors the synthetic live track at the LATEST run-begin
  marker in the buffer, and chapters are normalized so only the newest may
  be open-ended. Why: with rapid dev restarts several runs die between
  chapter-cache refreshes; anchoring at the first stray event merged them
  all into one track (Trace showed every run combined).

- Seek reuse ladder (serve / extend-left / extend-forward / flush+refill)
  with the anchor clamped to the stream floor. Why: every seek used to
  flush and refetch, blanking all panes for a frame on a one-pixel nudge.

UI (shared-ui components + project module)

- Needle PlayBar: fixed needle, timeline slides underneath, wheel zoom,
  thin-until-hover strip, grab-the-tape scrub, Shift-brush Analyze slice,
  day-grouped run menu with real clock times; transport is
  [Play|Pause] [speed] [|<] [<<] [>>] [>|] [readout] [GO LIVE], with all
  replay-only controls (and GO LIVE itself) hidden while live. Why: the
  previous bar could not navigate hundreds of runs across days, and live
  mode showed controls that had no meaning at the head.

- Status pane: fixed one-minute chart (range toggle removed — wider spans
  are the play bar's job), real clock axis labels ('Now' only when actually
  live), stats footer fed by trackStats, 450px pane body with the footer
  pinned and the chart centered.

- Trace: durations always render as seconds with two decimals (raw float
  millisecond stamps reached the screen before); tool-lane renderers show
  tool name/input and duration/output; scroll containment fixed (the pane
  fills the body exactly and only the tree scrolls — previously two
  scrollbars fought and tall traces were clipped by a non-flex wrapper).

- Log pane: Download Log (text projection) and Download Events (raw JSONL
  of the run) actions; breathing room below the terminal box; empty state
  replaces the frame instead of nesting inside it.

- Uniform EmptyState (the stock shared-ui placeholder) across
  Tokens/Flow/Trace/Errors/Log/Analyze. Why: six panes had six different
  ad-hoc empties; the style guide's control makes them one system. Flow's
  Pipeline/Component toggle hides when there is no data to toggle over.

- Flow pane folds pipeflow.byPipe from the track's flow events at the
  position. Why: status snapshots sample ~5s apart and documents finish in
  under a second, so sampled byPipe was almost always the empty idle
  instant.

Verified: full ai pytest suite green, tsc clean, saas + rocket-ui builds
green, live e2e against real runs (ingest, replay, seeks, tool calls,
multi-run restarts).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(shared-ui): SourceSection on the stock Card; kill the bespoke micro-button tier

Stock components exist so every application looks and behaves the same — a
hand-rolled duplicate of a stock component is a defect. This lands the first
conversion of the stock-components adoption plan and removes the style tier
that triggered it.

- PipelineActions: the "Chat now" / "Endpoint Info" buttons are now stock
  <Button small variant="primary|secondary">. The private micro-button tier
  (btnSmBase + primaryBtn/secondaryBtn + the --rr-btn-sm-* sizing, plus the
  unused endpoint-row styles) is DELETED. Why: it was a bespoke button
  system living inside one component; if a smaller button size is ever
  genuinely needed it must be a Button prop, never a per-component style
  block. Clicks stop at the container so canvas node presses never bubble
  into node selection/drag.

- StatusHeader: split into composable pieces — StatusHeaderInfo (state dot +
  name + state + status subtitle) and StatusElapsed ("Started Xs ago") — so
  Card-based hosts can place them in the Card's header/headerActions slots
  instead of nesting a second header row. StatusHeader remains as the
  self-contained composition for non-Card hosts. StatusActions now renders a
  stock <Button small> (danger for Stop, primary for Run/Subscribe) per the
  Card-header rule that header buttons are plain Button small.

- SourceSection: the section shell is now a real stock Card —
  StatusHeaderInfo in the header slot, the action cluster + elapsed in
  headerActions, the pill bar as the Card toolbar, and the pane body +
  PlayBar as an edge-to-edge body (noBodyPadding). Why: the section was
  spreading commonStyles.card and hand-building everything the Card
  component already owns (header row, divider, actions cluster), which is
  exactly how the Endpoint Info button ended up bespoke in the first place.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(vscode,shell-ui): task arguments + debug output become per-task pipeline settings

Move "engine arguments" and "debug output" out of the per-connection-group
local-engine settings and re-introduce them as per-task pipeline settings
(taskArguments, pipelineDebugOutput) applied through `.use` at task start.

Why: the old knobs were ENGINE PROCESS flags — changing tracing or adding an
argument meant tearing down and respawning the local engine, and the settings
were meaningless against a cloud connection (no local process to flag). But
both knobs are really properties of a RUN, not of the engine: the engine
already accepts per-task args via `.use`, and --trace=debugOut belongs to the
task being debugged. Modeling them per-task makes them work identically on
local, on-prem, and cloud connections, and takes effect on the next run
instead of the next engine restart.

Details:
- config.ts: taskArguments + pipelineDebugOutput join the pipeline settings
  group (with getTaskArgs()); the local connection group keeps only
  engineVersion. The raw argument string is intentionally passed as a single
  entry — the engine shell-splits it, so quoted paths survive.
- engine-local.ts: the engine now spawns with base args only; no per-task
  flags leak into the process command line.
- debugger/session.ts: launch requests attach getTaskArgs() so F5 runs get
  the same per-task arguments as panel-started runs.
- extension.ts: the engineArgs array->string migration and the old
  local.debugOutput/engineArgs key migrations are deleted with the keys
  themselves — there is nothing left to migrate into.
- package.json/PipelineSettings/SettingsWebview/panels: the settings UI moves
  the two fields into the Pipeline section, adds enum descriptions and
  explicit ordering so the webview and the contributes.configuration model
  stay mirror images of the cloud app manifest.
- shell-ui: SettingSchema documents that `enum` stays string[] per the frozen
  v0 contract while integer schemas (pipeline TTL) may carry numeric entries
  in manifest JSON at runtime — renderers go through String() and coerce back
  via `type`, which is what SettingsPage now does.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(shared-ui): stock Button gains the mini (16px) tier; canvas node actions use it

The stock-Card migration replaced PipelineActions' private micro buttons with
Button small (26px). That is correct in card headers but overflows the canvas
source-node card: "Drop now" + "Endpoint Info" side by side are wider than
the node. Per the rule that killed the micro tier in the first place — a
smaller button size must be a Button prop, never a per-component style block —
the old sizing (16px tall, 9px font, 0 6px padding, radius 3, weight 500)
returns as a stock `mini` prop, and PipelineActions renders mini in both of
its hosts. `mini` composes after `small` so it wins if both are set.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(run-logging): DVR v2 — client log sessions, permanent trace identity, live/replay parity

One coherent layer replaces the ad-hoc "subscribe + backfill" model: every
consumer of task events now reads from a LogEventStream SESSION over the
run-log continuum, and live delivery is just the last stop of the same walk.

== Client session (log-stream.ts / log_stream.py, both SDKs in lockstep) ==

- Live events are NOT a special case. Arrivals append to a "live bucket" that
  is an ordinary CachedSegment with a sentinel id sorting after every server
  segment, so history walks, trace collection, and status folds all traverse
  disk segments and the live tail through one code path. INVARIANT: the live
  bucket is never discarded — only prefixes proven to be server-backed leave
  it. When the bucket exceeds 24MB the catalog is reloaded and the prefix
  covered by sealed segments is dropped; extending the last disk segment
  advances maxDiskSeq the same way. Why: any policy that could throw away a
  live event that never reached disk silently loses data at exactly the
  moment the user is watching it happen.
- Two delivery modes, no pump/poll machinery: HISTORY walks the cached
  segments in order until exhausted, then flips to LIVE; in LIVE mode an
  arrival appends to the bucket AND is delivered directly to the session
  callback (when pinned + playing + past the watermark). The walk cursor
  dedupes the overlap by seq. Why: the previous arrival-drop/poke iterations
  had windows where events sat in the bucket with no deliverer — the
  two-mode model has none, and disk state is irrelevant to live delivery.
- Segment cache is capped at 6 resident segments, evicting least-recently
  USED (touched on ensure and on pump serve) — farthest-from-needle eviction
  would evict exactly the segments a replay is about to need again.
- Sessions self-register their monitor (all event classes) on open and
  deregister on close, refcounted. Why: consumers were silently missing
  OUTPUT/TASK-class events because the host's canvas-oriented monitor
  (summary/flow) was the only registration; the session knows what it needs
  and must not depend on what some host happened to subscribe.

== Trace identity (contract) ==

- A trace IS its begin event's continuum seq. Slot ids recycle within a run
  and fold docIds are synthetic per-client counters — neither survives a
  refresh, both collide across runs. The begin seq is epoch-microsecond
  seeded and globally unique for the life of the stream, so
  getTrace(beginSeq) is deterministic from any position, live or replay,
  in both SDKs: span-table binary search to the owning segment, then collect
  the slot's flow events forward to its end, crossing segments and the live
  bucket, with extend-on-miss for a still-active tail segment.
- SSE events ride along: the writer records which segments an in-flight
  document's SSE touched, and trace collection includes apaevt_sse rows by
  pipe_id, so node narration ("embedding chunk 3/10") appears inside the
  call tree it belongs to.

== Server (task_engine.py / run_log.py) ==

- _forward_task_event stamps project_id + source into every task-scoped body
  centrally, replacing per-event-builder stamping and envelope-id suffix
  tricks. Why: clients must be able to filter membership with a pure body
  test; identity established at ONE choke point cannot drift per event type.
- apaevt_task begin carries beginSeq — the run-begin lifecycle marker's seq
  (the marker itself is log-only). Clients synthesize the chapter locally
  from the announcement and join the server's chapter list by exact seq
  equality. Why: a task-begin-triggered chapter refetch is both a race (the
  control file may not be rewritten yet) and a thundering herd (every
  watching client fetching at once); carrying the identity in the
  announcement makes the local fix-up exact and free.
- Teardown order: the run log closes LAST, after the final zeroed status and
  the terminal task/lifecycle forwards. Why: the log is the record; closing
  it before the terminal events made every recorded run end without its end.

== UI (shared-ui project module) ==

- useTaskEvents consumes a session instead of listening to host events:
  fold with runway throttling, FOLD_CAP honesty, and DEAD ZONE = CLEARED —
  outside any chapter, every pane reports empty/zero at ANY position, live
  or replay. Why: stale panels in the gap between runs read as data from a
  run that never happened.
- |< seeks to the current chapter's begin (or the previous chapter's end)
  and >| to the chapter end (or next begin), with an epsilon so landing on a
  boundary resolves into the intended track.
- SourceSection owns the session lifecycle, keeps the local timeline fixed
  up from task begin/end announcements (server list reconciled by the slow
  poll only), and hosts the trace DetailPanel: right-docked, width persisted,
  Prev/Next over the request list, TraceDetail fetching by beginSeq.
- Trace pane becomes a pure request list; TraceDetail is a standalone
  component (also opened from Analyze) rendering the full call tree with
  Data|JSON|Raw stock ToggleGroup and interleaved SSE narration rows.
- AnalyzePane's slowest-documents rows open the trace popup instead of
  seeking — inspecting WHY a document was slow is the point of the table,
  and the begin seq each row already carries is the popup's exact key.
- LogPane caps its projection honestly (notices for both the display cap and
  the fold window) and downloads via a full continuum walk, so the file is
  complete even when the screen is not.
- PlayBar colors the live block from task begin/end announcements
  (runActive) rather than timeline completion guesses, with a square live
  edge.

== Tests ==

- test_log_stream.py: golden suite proving play==read equivalence, splice
  and console/status exactness, getTrace by begin seq across recycled slots,
  SSE inclusion, active-segment growth, direct live delivery, raced-fetch
  delivery, the 24MB split, and the LRU cap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(run-logging): address PR 1661 review findings — pump pacing, seam drain, epoch races, vscode event routing

Fixes for the verified review findings on the run-logging PR. Each carries the
why; two findings were rejected (reasons on the PR threads: the conftest stubs
are session-scope shims by design, and migrating engine-process args into
per-task args would inject semantically wrong flags).

- log-stream.ts / log_stream.py: timed playback (speed > 0) stalled forever on
  the first inter-event gap — the pacing branch slept/rescheduled and then
  recomputed the SAME delay from an unchanged playback clock. The clock now
  advances past the wait (epoch-guarded in TS so a retired timer cannot touch
  the live clock). The review caught the Python copy; the TS setTimeout
  re-entry had the identical stall and is fixed to match.

- log-stream.ts / log_stream.py: an event arriving during the history-walk's
  final awaits (nextEvent exhaustion -> refreshTimeline) was buffered into the
  live bucket but never delivered — ingestLive only delivers arrivals AFTER
  the pin flips, and the walk was already done. The pump now synchronously
  drains bucket events above the watermark right after pinning; no await can
  interleave an arrival mid-drain in either runtime.

- log-stream.ts: seek() now retires any in-flight pump (pumpEpoch bump) before
  its repositioning awaits — stopTimer only kills a SCHEDULED re-entry, so a
  pump parked on a fetch could resume against the old epoch and mutate
  watermark/basePos or fire a stale callback interleaved with the new
  position. Pump post-await guards also re-check playing/closed, so a pause
  landing mid-fetch cannot leak one extra delivery. (Python is immune: its
  seek cancels the pump task outright.)

- vscode ProjectProvider/ProjectWebview: the host monitor subscribed only
  summary/flow and forwarded only status/flow events, so the webview's live
  log and run-active timeline never received output/lifecycle events. The
  monitor now adds output+task, handleEvent forwards every stamped event for
  the editor's project, and the webview's membership check becomes the pure
  body.project_id test (the envelope source-suffix hack is deleted — the
  server's central identity stamp made it obsolete).

- vscode SidebarProvider: sidebar-started runs now pass the same
  pipelineTraceLevel + ttl as editor launches — a pipeline must run
  identically regardless of where it is started.

- shared-ui utils.ts: parseServerEvent skips flow events missing the continuum
  header stamps instead of emitting undefined eventTime/seq into the folds
  (NaN timestamps, unstable ordering). A skip, not a wall-clock fallback: the
  folds are event-time-sourced by design; against a pre-continuum server the
  trace pane is empty rather than corrupt.

- useTraceState.ts: the fold-reset effect reads resetKey but did not list it —
  the reset only fired because both callers change the traceEvents reference
  alongside it. The dep is added; the ref compare keeps re-runs idempotent.

- docs guide/methods/log.md: both getTrace examples passed the recycled pipe
  slot (.id) instead of the trace identity (beginSeq), contradicting the
  contract stated in the same file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(run-logging): the DAP envelope is protocol again — continuum stamps live in the body, and only there

The continuum seq was riding in the DAP header `seq` slot. That violates
DAP: `seq` is one monotonic counter per connection, shared by requests,
responses, and events on that socket. With per-task continuum counters, a
client monitoring two tasks interleaved two unrelated sequences (event Y
followed by event Y-555), and even one task mixed the connection's small
counter with continuum values. Correlation (request_seq) was never broken,
but event-stream sequencing was broken protocol.

The fix draws a hard line: the DAP envelope is TRANSPORT, the body is the
RECORD — and the record has exactly ONE representation.

- dap_conn.py is byte-identical to develop — send_event(event, id, body),
  nothing else. A generic DAP handler carries no logging concepts; the
  branch's earlier stamp-forwarding parameters are gone. Every connection
  mints its own envelope seq; the monitor forward is plain send_event, and
  the shared body carries the stamps to every subscriber identically.
- stamp_log_event writes body.eventTime + body.logSeq, beside the
  project_id/source identity already stamped there. Nothing nonstandard
  ever touches the envelope: eventTime as a header field was itself a
  spec extension, so both stamps moved into the body together.
- NO mirroring anywhere: clients read body.eventTime/body.logSeq directly.
  The SDK LogEvent (TS + Python) and shared-ui's TaskEventMessage lost
  their top-level stamp fields — the body is typed (LogEventBody /
  TaskEventBody) with the stamps required, and every internal walk
  (watermarks, pacing, binary searches, folds, getTrace, seam drain,
  maxDiskSeq) reads the body. Why no mirror: a second copy of the stamps
  is a second thing that can be wrong, and the mirrored type read like a
  wire claim — the type said "header eventTime" while the wire said
  otherwise. One representation cannot drift.
- Legacy compatibility is normalization INTO the body: codec decode
  canonicalizes old header-stamped v2 segments (body.eventTime/logSeq
  filled from the header once), so old recordings read identically to new
  and no consumer carries a fallback.
- Host gates (rocket-ui ProjectProvider, vscode ProjectWebview) admit an
  event on one pure body test — identity + stamps present — and pass the
  message through untouched.
- Seeding is catalog-based: a fresh stream starts at 1; open() lifts the
  floor to control.lastSeq + 1, so the continuum continues exactly where
  the recorded stream left off. The epoch-microsecond seed, the max(clock,
  lastSeq) logic, the clock-anomaly marker, and CONST_LOG_SEQ_FLOOR are
  deleted. Why: the writer reads the control file at open anyway, so the
  monotonicity guarantee is free, and small human-readable seqs beat
  16-digit epoch values. The crash window (an unpersisted tail re-issuing
  values) is accepted: a crash drops every websocket, clients reconnect
  with fresh sessions and fresh live buckets, so nothing stale survives to
  collide.
- shell-api cleared and REFROZEN at v0 via `./builder shell:freeze`
  (sanctioned: nothing has been released, so no consumer ever saw the old
  contract). The frozen LogEvent is the body-only shape; SHELL_API_VERSION
  restamped to 0; all accumulation files regenerated by the freeze script.
- Tests: the fake stamp mirrors body semantics; golden assertions read
  through body accessors (seq_of/time_of); the clock-anomaly test dies
  with the behavior it tested; fresh-stream seqs assert from 1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(run-logging): live-minimal PlayBar, RocketRide banner in every run log, terminal-tight log lines

Three polish items from live testing:

- PlayBar: pinned-live-and-hands-off is now just the bar — the pause
  button, the LIVE clock readout, and the zoom scale fold into the hover
  (which also unfolds the full lane, as engagement always has). Replay
  counts as engaged, so its chrome never hides. The collapsed live bar is
  a little taller (16px vs the 10px replay ribbon): with the chrome gone,
  the bar is the only presence at the live edge, so it gets more of one.

- The RocketRide ASCII banner now opens every run's log. It lived only in
  ai/web's WebServer.serve() — the EaaS operator console — and the task
  node never runs a web server (node.py hands the process to rocketlib),
  so the run log never saw it. The art moves to a shared ai/logo.py;
  WebServer imports it and node.run() prints it as its first act on
  stdout, which IS the console the run log records.

- Log pane line height 1.55 -> 1.25: log output (and the banner art)
  reads as a terminal, not prose.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix(run-logging): address PR 1661 review round 2 — finite stamps, span keying, legacy read() canonicalization, DBG label safety

All fourteen review findings verified and fixed (none rejected this round):

- Stamp guards are Number.isFinite / math.isfinite everywhere a wire event
  is admitted: parseServerEvent, both host gates (rocket-ui ProjectProvider,
  vscode ProjectWebview), and BOTH SDK ingest paths — typeof admits
  NaN/Infinity, and Python's ingest additionally required only logSeq, so a
  time-less event would have folded at timestamp 0 into every seeded read.

- AnalyzePane keys open enter/leave stacks by slot:component. Concurrent
  documents run the same components on different slots; a component-only
  stack paired one document's enter with another's leave and corrupted the
  latency totals. Reentrancy within a slot still stacks, like the trace fold.

- The server reader canonicalizes legacy header stamps INTO the body before
  returning events, so everything read() yields satisfies the body-stamp
  contract — consumers never carry a header fallback. This matches what the
  session codec already did on the segment path.

- DBG tool-trace labels can no longer break the wire format: '*' is the DBG
  field delimiter and only the final JSON field tolerates it (maxsplit), so
  the no-node call label is now a bare 'tool()' (no placeholder — reads
  naturally in the trace tree) and _emit_tool_trace strips '*' from the
  component field defensively. The wire format itself is unchanged (shared
  with the C++ debugger).

- vscode migrations fold the four legacy *.debugOutput keys into the one
  per-task pipelineDebugOutput (OR per scope — any scope that had it on
  keeps it on) and clear the legacy keys. engineArgs stays unmigrated:
  engine PROCESS flags are not per-task arguments, and injecting them into
  every .use call could break runs.

- LogPane downloads attach the anchor to the document for the click and
  defer the blob-URL revoke — a synchronous revoke can cancel a download
  that has not begun fetching (same pattern as the data-grid export helper).

- summaryToolInput only renders no-args for absent input or an empty plain
  object — falsy scalars (0, false, '') are real arguments and show as such.

- SettingSchema.enum widened to Array<string | number> (workspace types +
  app-sdk) — integer schemas carry numeric choices; rendering keeps the
  String() coercion.

- Docs: client.log gains its co-located Python guide
  (client-python/docs/log.md); the getTrace failure contract in both SDK
  sessions now names both failure modes (below horizon, or no trace-begin
  at that seq); TraceEvent.seq loses the stale epoch-us wording.

- shell-api v0 cleared + refrozen via `./builder shell:freeze` (pre-release
  procedure) — the frozen surface picks up the widened enum, the
  canonicalized read() contract, and the corrected getTrace docs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(vscode): debug-output migration gets its own marker and loss-proof ordering

Review findings on the migration added in the previous round:

- It lived inside the settingsMigrationV2Done block, so any user who had
  already completed the V2 migration would skip it forever. It now runs
  under its own debugOutputMigrationDone marker, independent of V2.
- It deleted the legacy keys while collecting them, before writing the
  replacement — a failure mid-migration would have destroyed the only copy
  of the old configuration. The order is now collect, write
  pipelineDebugOutput, then delete the legacy keys: retry-safe at every
  step.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(vscode): remove the Rocket Ride: Console output channel

The channel mirrored every server output event into VS Code's Output
panel. The task's Log pane now renders the same events (run-log backed,
live and replay), so the mirror was a redundant second copy with none of
the log's structure. Removal, end to end:

- connection.ts no longer writes output events to the channel; the
  shell:event emit that feeds the webview panes is untouched, and
  non-output apaevt_* diagnostics still log to the main Output channel.
- The channel and logger.console() are deleted; engine spawn/readiness
  lines (engine-process diagnostics, not task output) move to the main
  Output channel.
- The sidebar's wildcard monitor narrows from ['task','output'] to
  ['task']: its output subscription existed solely to feed the mirror, so
  the server no longer broadcasts every console line to the extension for
  a consumer that no longer exists. Per-editor monitors subscribe to
  output themselves for the Log pane.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* style(docs): ruff format markdown code blocks to unblock CI

The Ruff gate runs `ruff format --check` with docstring-code-format
enabled, which formats the python code fences inside our docs. 14 doc
files carried unformatted snippets (mostly pre-existing, not introduced
by this branch) and were tripping the check on this PR. No source code
changed — docs only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nTVr6jfSFYm1GppxbjghP

* fix(task,docs): accept the bare >JOB control (empty status); executable doc examples

- transport_stdio: a bare '>JOB' (no delimiter) is the empty job status —
  the engine's monitorStatus('') only appends '*text' when there is text,
  and nodes use the empty form to CLEAR the status line (e.g. the sentence
  transformer after model load). It now forwards as an empty
  apaevt_status_message instead of falling through to the unknown-control
  branch and dumping 'Unknown control: >JOB' into every run's console.

- Python SDK docs (review round 4): the index.md context-manager example is
  now executable (async main + imports + asyncio.run — async with cannot
  appear at module scope), and the log.md closed-trace example guards
  traces['closed'][0], which is legitimately empty on a fresh or
  all-in-flight stream.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Dmitrii Kataraev <dmitrii.kataraev@rocketride.ai>
AviNormie pushed a commit to AviNormie/rocketride-server that referenced this pull request Nov 9, 2026
* feat(nodes): add tool_pipedrive CRM tool node

Exposes the Pipedrive REST API v1 to agents: 256 tools across 24 resource
groups (deals, persons, organizations, activities, pipelines, stages, notes,
leads, products, fields, files, users, roles, permission sets, teams, goals,
filters, webhooks, subscriptions, mailbox, call logs, projects, misc), plus a
generic `request` tool that reaches any remaining v1 endpoint through the same
auth, rate-limit and read-only layer.

Full coverage is more tools than an LLM can choose between reliably, so
`pipedrive.toolGroups` controls which groups are published. The default eight
core CRM groups publish 108 tools; `["all"]` publishes everything. Filtering
happens in `IInstance._collect_tool_methods()`, so a gated tool is invisible to
`tool.query` and refused by `tool.invoke` alike.

Client details specific to Pipedrive:
- `{"success", "data", "additional_data"}` envelope is unwrapped; `success:
  false` on an HTTP 200 is raised as an error.
- `x-ratelimit-reset` is the seconds remaining in the window, not an epoch
  timestamp as in the GitHub client this otherwise mirrors.
- Offset pagination (`start`/`limit`, max 500) everywhere except projects,
  which use cursor pagination.
- Personal API tokens go in the `api_token` query parameter; JWT, long or
  `Bearer `-prefixed values go in an Authorization header.
- Custom fields are grouped under `custom_fields` on read and written through
  the `extra` passthrough that every create and update tool accepts.

Tests: 81 unit tests that need no credentials or network, plus an env-gated
live suite behind PIPEDRIVE_API_TOKEN, with record-creating tests additionally
behind PIPEDRIVE_ALLOW_WRITES.

Also corrects the stale node/service counts in docs/README-nodes.md
(88 -> 118 directories, 118 -> 148 services).

Closes rocketride-org#1675

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes): use the real Pipedrive brand mark for the tool_pipedrive icon

The first icon was a green uppercase "P" next to an unrelated dark dot. The
actual Pipedrive mark is a solid green disc carrying a white lowercase "p"
whose counter shows the disc through it.

Redrawn as plain geometry (disc r=128, stem 36x150, bowl r=56, counter r=27)
so it stays crisp at 16px with no gradients, clip paths or embedded raster.

Uses the default nonzero fill rule with the counter arc wound opposite to the
bowl. Under evenodd the stem and bowl would XOR where they overlap and punch a
green notch out of their junction; nonzero merges them solid while the opposing
winding still renders the counter as a hole.

Keeps the viewBox (SVGR runs with dimensions:false, so an icon without one
computes to zero width) and keeps two distinct colours, which makes the
auto-currentcolor svgo plugin treat it as a multicolour brand mark and pass the
authored colours through untinted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes): trace the tool_pipedrive icon from the supplied brand artwork

The previous version approximated the mark with circles. The real letterform
differs in three ways that are visible even at 16px: the stem carries a spur at
the top left, the bowl is an ellipse rather than a circle, and the counter is an
ellipse tangent to the stem's right edge.

Geometry was measured off the supplied 272x272 artwork rather than eyeballed —
connected-component analysis to isolate the counter, then a least-squares fit of
the bowl ellipse to the glyph's right-hand edge (max residual 2px). Rasterising
the traced primitives and diffing against the source leaves 1.47% of opaque
pixels differing, all of it a sub-pixel outline along the shape boundary.

Still a two-colour multicolour mark, so the auto-currentcolor svgo plugin passes
the authored colours through, and still nonzero winding with the counter wound
opposite the bowl so the stem and bowl merge solid while the counter stays open.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(nodes): warn when tool_pipedrive publishes too many tools for an agent

Full coverage is 256 tools. An operator can already reach that by selecting
groups, and nothing told them the agent's tool choice degrades long before it.

Adds a guard rail that counts published tools rather than groups, because group
sizes are lopsided: deals is 28 tools and permission_sets is 3, so a group count
bounds nothing useful. The threshold is 120, just under the 128-function cap
some providers impose on a single request and above the 107 the default groups
publish.

Crossing it warns and carries on. The tools are still published, in both
validateConfig (so it shows in the config panel while editing) and beginGlobal
(so it shows at pipeline start). Silently dropping tools an operator asked for
surfaces later as "the agent won't do X", which is worse than a noisy panel. The
warning is deliberately not raised from _collect_tool_methods, which runs on
every tool.query and tool.invoke.

"all" is exempt: it is already an explicit opt-in to the whole surface, and
scripted callers that never route through an LLM legitimately want it. Listing
every group by name is not exempt.

Group sizes are derived from the __pipedrive_group__ stamps rather than a
hand-maintained table, so a new tool cannot drift out of sync; a test asserts
every group in ALL_GROUPS has a count. Another test pins the advisory semantics
so the warning is not later "fixed" into a silent truncation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai): flatten provider content blocks to visible text

Anthropic with extended thinking returns `content` as a list of typed
blocks, not a string. Every non-streaming path handed that list straight
to callers, who stringified it — so the answer arrived as the blocks'
repr, carrying the raw reasoning and its base64 signatures.

For agents it was worse than cosmetic: the ReAct parser reads output back
with a greedy `Action Input: (.*)`, so the serialized wrapper's trailing
`", "type": "text"}]` landed inside the tool arguments and every declared
field then reported as missing.

Extract the block vocabulary that only the streaming path knew into
`ai.common.utils.flatten_content_blocks` and route chat's invoke, the
stream fallback, and the agent's `extract_text` through it. The streaming
path now shares that vocabulary instead of duplicating it, so the paths
cannot drift; the fallback also forwards recovered reasoning to
`on_reasoning_chunk` rather than dropping it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes): make CrewAI tool calls and hierarchical kickoffs survive

Three defects made the CrewAI nodes fail on runs that should have worked.

Tool arguments were dropped. `_ToolInput` declared a single `input` field,
and CrewAI's argument filter intersects the model's emitted keys with the
schema's property names — so every real parameter was discarded before
validation, which then reported all required fields missing. Generate a
typed model per tool instead: JSON Schema types map to real annotations
(so pydantic coerces "123" -> 123), optional fields accept an explicit
null, and paramless tools get an empty schema. The JSON schema is no
longer appended to the description either; CrewAI renders args_schema into
the prompt itself, and the two copies disagreed.

`HostInvokeLLM` never declared `supports_function_calling()`. CrewAI
defines it on `LLM` but not on `BaseLLM`, and several call sites invoke it
unguarded — converter.py, tool_usage.py, reasoning_handler, and
task_evaluator. The AttributeError was swallowed by converter's broad
except and re-reported as "Failed to convert text into a Pydantic model".
Declare it False: the host channel is text-in / text-out and cannot honour
native tool calls, so every call belongs on the ReAct path. Pin crewai <2
for the same reason — 1.15.x moves the BaseLLM contract.

Hierarchical crews then died in planning. `Crew(planning=True)` runs a
task with `output_pydantic` set and drives it with the host wrapper, which
cannot promise JSON; unparseable plan text dead-ends the converter and
raises "Failed to get the Planning output". Make planning opt-in via a new
**Crew Planning** field, off by default.

Also clear `Task.tools` after construction: CrewAI back-fills them from
the task's agent, and a hierarchical crew resolves the executing agent's
toolset for the manager — which let the manager work a delegate's tools
itself instead of delegating.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes): move tool_pipedrive search to the v2 API

Pipedrive retired the v1 search routes: `/persons/search` and its siblings
answer 404 "Unknown method ." and `/itemSearch` answers a plain 404, so
every search tool the node published was dead. Non-search v1 endpoints
still route normally, so only the search calls move — a blanket swap would
break the many endpoints with no v2 equivalent.

Adds a v2 base URL (company-domain aware, sharing v1's normalisation) and
a v2 envelope wrapper: v2 replaced offset pagination with an opaque
`next_cursor`, which `paginated` would have missed entirely and reported
as a single page. Search tool schemas now advertise `cursor` instead of
`start`, and `item_search_by_field` follows v2's renames — `field_type` ->
`entity_type` with the "Field" suffix dropped from each value, and
`exact_match` replaced by the three-way `match` mode.

Also turns Tool groups into a multi-select dropdown with per-group tool
counts, so the field no longer requires typing group names from memory.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(shared-ui): keep live panes working without a host event stream

PR rocketride-org#1661 rewired every accumulating pane — Trace, Errors, Status, Tokens,
Flow, Log, Analyze — to read through a DVR session obtained from the
host's `openEventStream`, a prop no host in this repo binds. Hosts that
have not bridged it across their transport (the VS Code webview is one)
passed a null session, `useTaskEvents.ingestLive` dropped every event, and
the panes rendered their empty states mid-run with no error anywhere.

Fall back to an in-memory store over the live events the host already
delivers. It satisfies the same session contract `deliver` expects — a
pure append, each event handed over exactly once, oldest first, across
repeated `play` calls — and is keyed per stream identity so the several
views `SourceSection` opens all share one buffer. Replay of recorded runs
still requires the real stream.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(test): stop node packages being shadowed by same-named test dirs

nodes/test is a package whose subdirectories are named after node packages
(text_output/, response/, image_cleanup/, telegram/, tool_git/, ...). Two places
put that directory on sys.path so they could bare-import a test-root module:
conftest (for _sys_modules_guard) and cloud_tts/test_cloud_tts (for
test_contracts). Once nodes/test sits ahead of src/nodes, `import text_output`
resolves to the *test* package, whose __path__ holds no node module — so
`from text_output.instance import Instance` dies with ModuleNotFoundError.

Whether it bites was pure collection order. The affected modules bootstrap with
"insert src/nodes only if absent", which never re-prioritises: previously nothing
seeded src/nodes before cloud_tts (c) inserted nodes/test, so image_cleanup (i)
was the first to insert src/nodes and landed it in front. The new agent_crewai
tests seed src/nodes during collection of agent_crewai (a) instead, so cloud_tts
now stacks nodes/test on top of it and the guard short-circuits in all four later
modules — four collection errors, none of them in code this branch touches.

Import both test-root modules package-relative and drop the sys.path inserts, so
nodes/test never shadows a node package. cloud_tts gains the __init__.py the rest
of the node-named test dirs already have, which is what makes the relative import
resolvable. The four bootstraps now move src/nodes to the front unconditionally
rather than skipping when it is already somewhere on the path, and the guard suite
pins both invariants: nodes/test stays off sys.path, and a node package imported
by name resolves under src/nodes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes, shared-ui): address CodeRabbit review on the tool_pipedrive PR

Security
- pipedrive_client._company_host accepted anything up to the first slash, so a
  configured domain of "evil.example#" built https://evil.example#.pipedrive.com
  and Requests sent it — with the api_token or Authorization header attached — to
  evil.example. Cut at the first path/query/fragment delimiter, refuse userinfo
  and ports, lower-case, and require a single DNS label; anything else falls back
  to the generic host.
- Free-form ids (lead uuids, permission-set ids, channel ids, goal ids, note
  comment ids) were f-stringed into request paths unencoded, so a value carrying
  "/", "..", "?" or "#" retargeted the call at another endpoint — for the DELETE
  tools, a destructive one. Added _base.path_segment() and applied it at all 20
  string-id sites.
- Pin requests>=2.34.2 to match constraints.lock, as tool_cognee/tool_github/
  tool_n8n already do.

Correctness
- crewai_base._annotation_for passed prop['type'] straight to dict.get. JSON
  Schema spells a nullable field as ["string", "null"], and an unhashable key
  raises TypeError instead of returning the default — one such property aborted
  the whole _build_crew_tools loop. Non-str types now fall back to Any, which is
  what the docstring already promised.
- deal_followers_users_list GET the same /deals/{id}/followers as
  deal_followers_list and only differed by running follower rows through
  clean_user, emitting a user-shaped object built from mismatched fields.
  Dropped it; resolving followers to users needs a /users/{id} call per follower.
- add_time (deals, organizations, persons) and origin_id (leads) were in the
  write-key tuples but not in the matching schema props, so body_from could never
  populate them. Added the properties.
- product_variation_update and both project-plan updates passed the builtin dict
  as the response cleaner: dict(None) raises TypeError on an empty 204 body.
  Switched to passthrough, as every other write already uses.
- liveEventSession.getStatus scanned from the live edge, contradicting the
  documented "as of the session position" and the caller, which seeks to an
  anchor first — a repositioned view seeded a replayed run with the current
  status. Scan back from the watermark.

Maintainability
- Eight bulk deletes each repeated the ids guard, the CSV build and a trailing
  _require_write(), and left ids optional so a missing argument surfaced as a
  hand-rolled ValueError. Added _base._delete_bulk (write gate first), marked ids
  required in every schema, and delegated all eight.
- projects._cursor_list re-implemented paging_params_v2 and the v2 envelope
  shaping; it now uses the shared helpers, so the cursor contract has one
  implementation instead of two that can drift.
- file_list advertised the shared 1-500 range, but /files documents a maximum
  limit of 100. PAGING/paging_params/_list take a max_limit override and file_list
  passes 100.
- _build_crew_tools' docstring still claimed the JSON Schema is embedded in the
  tool description, which stopped being true when args_schema took over.
- Tests: 'ACME'.lower() evaluated to 'acme' before the call, so the case path was
  never exercised; test_tools imported pipedrive_client bare, giving the module a
  second identity in a run that also collects test_pipedrive; a vacuous
  b64encode assert, a missing empty-pipelines guard, and a hard-coded field_id
  of 1 that fails rather than skips on a fresh sandbox.

Two findings were not applied. `extra` is not dropped by fields/misc — body_from
merges args['extra'] by default (tools/_base.py). And the @patch/parametrize
signatures are already correct: mock injects positionally innermost-first while
pytest passes parametrize values by keyword, so the suggested reorder would bind
the mocks to the parametrized names.

New tests cover the rejected domains, path-segment encoding, non-str schema
types, the write-gate ordering, and the watermark-bounded status.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes): harden the tool_pipedrive path and bulk-delete helpers

Two follow-ups from the second CodeRabbit pass, both on helpers added in the
previous commit.

_delete_bulk merged the caller's `extra` pass-through after assigning the
validated `ids`, so `deal_delete_bulk({'ids': [1], 'extra': {'ids': '2,3'}})`
deleted 2 and 3 instead of 1. Merge `extra` first and let the validated CSV win.
The hand-rolled version this replaced had the same ordering, so the flaw predates
the helper.

path_segment relied on quote() alone, but "." and ".." contain nothing quote
escapes — `/notes/{id}/comments/..` survived encoding intact and addressed the
parent resource. Reject those two segments instead of encoding them.

Tests cover both: a dot-segment id raises before any request is made, and a
pass-through "ids" no longer displaces the validated list while other extra
parameters still reach the query string.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(nodes, ai): keep the Pipedrive token out of transport error text

The personal-token path puts the credential in the `api_token` query parameter,
so requests bakes it into the URL. A transport-level failure (DNS, refused
connection, TLS, timeout) raises a RequestException whose text carries that URL,
and `raise ValueError(f'Pipedrive request failed: {exc}') from exc` copied the
token verbatim into the error the agent framework surfaces and logs. The chained
cause held the same URL, so scrubbing the message alone would still have leaked
it through any printed traceback.

Build the message from the exception type plus method and path — all secret-free,
and the type still distinguishes ConnectTimeout from SSLError from ConnectionError
— and raise `from None`. Verified this was the only such site: the JSON-decode
path stringifies `resp.text[:200]` rather than the URL, `_raise_pipedrive_error`
builds from parsed response fields, and the tenacity wrapper uses `reraise=True`
so no RetryError repr is introduced.

Three tests pin it, and all three fail against the previous code: the token
appears in neither the message nor the formatted traceback (that half is what
pins `from None`), and the message still names the failure and the call.

Also:
- notes' `add_time` was in _NOTE_WRITE_KEYS but not _NOTE_WRITE_PROPS, so
  body_from could never populate it — the same class fixed for deals,
  organizations, persons and leads in 7b3cc53. A scan of every
  *_WRITE_KEYS/*_WRITE_PROPS pair confirms notes was the last one.
- ChatBase._chat had no direct test. flatten_content_blocks and extract_text are
  covered, but nothing pinned the wiring that routes the provider's content
  through them, which is the contract every non-streaming chat caller depends on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
AviNormie pushed a commit to AviNormie/rocketride-server that referenced this pull request Nov 9, 2026
…mented ingester protocol (zero engine changes) (rocketride-org#1612)

* feat(sdk): rocketride.otelbridge — OpenTelemetry export over the documented ingester protocol (rocketride-org#1609)

Maps the engine's monitor event stream to standard OTel telemetry with
zero engine/server changes — the client-side refinement invited by
tracking issue rocketride-org#616 after the engine-invasive rocketride-org#498 was archived.

- mapper.py: stateful FlowSpanMapper reconstructs span trees from
  apaevt_flow — task root span, pipe span per pipe id, component child
  spans paired by component identity (per the observability doc's
  warning against stack-position pairing); trace.error -> span status
  ERROR + exception event; apaevt_sse -> span events; unclosed spans at
  end/shutdown flagged rocketride.span.unclosed=true; per-pipe state is
  bounded. MetricsMapper maps apaevt_status_update snapshots to OTel
  instruments (object counts, rates, cpu/gpu memory + peaks).
- Semantic conventions per the July 2026 semantic-conventions-genai
  snapshot: gen_ai.provider.name (not the deprecated gen_ai.system),
  gen_ai.operation.name from the well-known list only, span names
  '{operation} {model}' style, INTERNAL span kind for in-process
  component executions.
- Privacy by default: lane payloads / prompt / result content never
  reach attributes or events unless include_content is set (8KB
  truncation cap when it is); enforced by a sentinel-scan test across
  all exported attributes and events.
- setup.py: TracerProvider/MeterProvider + OTLP http/protobuf exporters;
  ALL opentelemetry imports are lazy so the base package imports without
  the extra.

53 unit tests (InMemorySpanExporter/InMemoryMetricReader; importorskip
so the base CI matrix stays green), including a full replay of the
captured real-engine event fixture.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(cli): add 'rocketride otel' subcommand + optional [otel] extra (rocketride-org#1609)

rocketride otel [--endpoint URL] [--protocol http|grpc] [--service-name]
[--headers k=v,...] [--include-content] [--no-metrics] + the standard
connection args. Subscribes via rrext_monitor with token '*' (the
observability doc's recommended ingestion scope, types TASK|SUMMARY|
FLOW|SSE), dispatches to the mappers, reconnects with capped backoff
and resubscribes (subscriptions are per-connection and not durable),
and flushes exporters on SIGINT/SIGTERM.

Endpoint resolution honors the standard OTLP env contract: explicit
--endpoint > OTEL_EXPORTER_OTLP_TRACES_ENDPOINT (verbatim) >
OTEL_EXPORTER_OTLP_ENDPOINT (base URL + /v1/traces|/v1/metrics) >
default. Packaging: opentelemetry-sdk + OTLP http exporter live behind
an optional extra — the base install gains zero dependencies, and
running the subcommand without the extra exits 2 with the exact
"pip install 'rocketride[otel]'" hint before any connection attempt.

25 tests: config precedence matrix, dispatch/reconnect/resubscribe with
a fake client, shutdown flush ordering, missing-extra exit path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: OpenTelemetry bridge guide with Jaeger, Langfuse, and Datadog recipes

Setup guide covering the span model (task -> pipe -> component tree),
the privacy default and --include-content, the FLOW-requires-trace-level
callout, reconnect semantics, and copy-paste recipes with exact OTLP
endpoints/auth for Jaeger all-in-one, Langfuse (/api/public/otel, Basic
auth), and Datadog. Notes that Jaeger does not ingest OTLP metrics —
pair it with --no-metrics or point metrics at a collector.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(sdk): handle partial OpenTelemetry installs — guard the HTTP exporter import

CI exposed a real environment our matrix missed: opentelemetry-api/sdk
present as another package's transitive dependency (tool_daytona pulls
them in) while the OTLP HTTP exporter is absent. Two fixes:

- setup.py: the http trace/metric exporter imports are now guarded like
  the grpc ones — a partial install raises OtelNotInstalledError with
  the pip install 'rocketride[otel]' hint (CLI exit 2) instead of a raw
  ModuleNotFoundError. Verified live in a reproduced partial venv.
- test_otel_setup.py: importorskip targets the most specific module the
  tests exercise (opentelemetry.exporter.otlp.proto.http), so the module
  skips cleanly in CI's partial environment; plus a regression test that
  simulates the partial install via a None sys.modules entry and asserts
  the friendly error.

Partial venv: 63 passed + 1 module skip; full extra: 79 passed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(sdk): OTel bridge review round — privacy-gate errors, run-id promotion, header precedence, shutdown ordering

CodeRabbit review round on rocketride-org#1612, each finding verified before acting:

- mapper: trace.error no longer bypasses the privacy gate — by default
  only the error signal exports (status ERROR, bare exception event,
  generic description); --include-content restores verbatim text, now
  subject to the same 8KB truncation strings previously skipped. The
  sentinel privacy test plants the sentinel in trace.error.
- mapper: a run first seen via flow events (before the task 'running'
  snapshot) is now promoted from its project|source fallback key to the
  canonical run id when it arrives — previously a second run state was
  created (broken parentage) and the orphaned fallback state leaked
  permanently. Confirmed with a failing repro; guard test ensures two
  distinct canonical runs sharing (project, source) never merge.
- config/setup: only --headers become explicit exporter headers; env
  headers are left to the SDK so OTEL_EXPORTER_OTLP_TRACES_HEADERS /
  _METRICS_HEADERS signal precedence works (it was silently defeated).
  Precedence matrix tests added.
- setup: meter provider shutdown wrapped in finally so metrics flush
  even when tracer shutdown raises.
- CLI: --trace-level none message now says flow tracing stays disabled;
  docs recipes pass collector auth via OTEL_EXPORTER_OTLP_HEADERS env
  instead of argv (shell-history hygiene); error-mapping doc wording
  matches the implementation (component leave events).

Full extra: 90 passed. Partial install (api/sdk without exporter):
69 passed + clean skip. ruff clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(sdk): absorb stranded fallback run state when canonical state precedes it

Follow-on to the run-promotion fix: an SSE event creates the canonical
run state WITHOUT registering the (project, source) alias (SSE bodies
carry no project/source per the protocol), so an id-less flow for the
same run could still open a parallel fallback-keyed state — and because
the canonical state already existed, the promotion branch was skipped,
stranding the fallback state (kept alive forever by snapshot
reconciliation) with duplicate spans.

_get_run now reconciles whenever an event carries both the canonical id
and project/source: no canonical state → promote (unchanged); canonical
state exists → _absorb_fallback_run migrates non-colliding pipes (open
spans re-stamped with the canonical run id), closes colliding pipes as
rocketride.span.unclosed=true, and drops the fallback key. Only
fallback-format keys are touched, so distinct canonical runs never
merge (guard test unchanged and green).

Both new regression tests failed before the fix. Full extra: 92 passed;
partial install: 71 passed + expected skip; ruff clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(sdk): stamp OTel spans with the engine's body.eventTime

The bridge was written against a monitor protocol that carried no event
timestamps, so every span used the bridge's own clock at event arrival and
the guide documented that as a limitation. Since rocketride-org#1661 the engine stamps
`body.eventTime` (epoch seconds, set once at ingress by the run-log
continuum) on every forwarded event, and derived flow events inherit their
source event's stamp — so the limitation is no longer true and the
documentation shipped with it was wrong against develop.

Span start/end and span-event timestamps now come from `body.eventTime`,
making durations engine-measured instead of bridge-observed: WebSocket
transport latency and event-loop scheduling no longer inflate every
component span.

The fallback is exactly the previous behaviour. A body with no usable
`eventTime` (an older engine, or a non-float/NaN/non-positive value) and
every span closed outside an event — shutdown, seeded-snapshot
reconciliation, tracked-run cap eviction — use `time.time_ns()`, the same
clock the SDK would have used. Because a run can therefore mix both
sources, each open span records its own start and a close is clamped to
never precede it, so no span can report a negative duration.

Metric points are unchanged: OpenTelemetry metric instruments take no
explicit timestamp, so they remain arrival-timed. That is now stated
rather than left implicit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(client-python): publish the OTel bridge guide and mirror its CLI entry

Two co-location gaps (AGENTS.md:5-28) and one stale claim.

The 369-line guide lived at docs/README-otel-bridge.md, and root docs/ is
not in the docs-site glob (packages/docs/scripts/lib/gather.js:25), so the
guide was GitHub-only. Moved to packages/client-python/docs/otel-bridge.md,
the tree already mounted at develop/python (packages/client-python/scripts/
tasks.js:236) alongside analytics.md and log.md, so it is gathered to the
site at /develop/python/otel-bridge. Its in-repo links become site routes
to match the sibling pages; the three references to it were repointed.

packages/client-python/docs/index.md is the site's Python page and mirrors
the PyPI README's ## CLI block line for line, but had no `rocketride otel`
row and none of the OpenTelemetry paragraph the README gained. Added both.

The guide's ## Limitations still said span timestamps are arrival time
because the wire carries none; the previous commit made spans engine-timed,
so that bullet now describes what actually remains bridge-timed (spans
closed outside an event, pre-continuum engines, and all metric points).

The Python blocks in the guide are also reformatted: ruff formats Markdown
code blocks as of 0.16, and CI's ruff-action is unpinned, so the file failed
`ruff format --check` today at either path. Repo-wide check is clean again.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(changelog): record the rocketride otel bridge under [Unreleased]

CHANGELOG.md [Unreleased] ### Added is the convention for user-visible
additions and is kept alphabetical by scope; a new CLI command and a new
optional extra qualify.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(client-python): name Grafana Tempo in the OTel backend recipes

Issue rocketride-org#1609's acceptance list says "Jaeger/Tempo", and the guide covered
Tempo only implicitly as "Grafana stacks". Tempo is the OTLP receiver a
Grafana user actually points the bridge at, and it needs one non-obvious
step (enable distributor.receivers.otlp) plus a separate metrics backend,
so it is worth naming rather than leaving a reader to infer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(sdk): address CodeRabbit review round on the rebased head

Nine threads on the rebased head, all in the OTel bridge. Dispositions:

- docs/otel-bridge.md:151 (Sensitive Data Exposure, CWE-522) — FIXED. The
  Langfuse and LangSmith recipes now prompt for the keys (printf + read -rs,
  spelled identically in bash and zsh) instead of embedding placeholders that
  operators replace with live secrets in a command line, and point at a secret
  manager as the better source. The base64 pipeline is now
  `printf '%s' ... | base64 | tr -d '\r\n'`: real Langfuse keys make the
  joined string 85 characters, past GNU base64's 76-character wrap, and
  command substitution strips only trailing newlines, so the embedded break
  would have produced an invalid Authorization header.

- cli/commands/otel.py:207 (Sensitive Data Exposure, CWE-532) — FIXED. The
  startup line ran `config.endpoint` straight to stdout; an endpoint can carry
  userinfo credentials or a signed query string. It is now always passed
  through `redact_endpoint()`, which drops userinfo, query and fragment while
  keeping scheme/host/port and the ingest path operators actually verify.

- cli/main.py:466, otelbridge/config.py:177, otelbridge/setup.py:123
  (Cleartext Transmission, CWE-319) — FIXED, once, in one place.
  `validate_transport_security()` runs before any exporter exists (in
  `build_providers`, and again in the CLI so the failure is exit code 2 rather
  than a traceback): a credential-looking header name from `--headers` or any
  of the three OTEL_EXPORTER_OTLP_*HEADERS variables, plus a non-loopback
  cleartext endpoint for either signal, aborts startup. Header VALUES are
  never read or echoed — only names reach the message. Loopback stays exempt
  for local collectors; `--insecure` / ROCKETRIDE_OTEL_ALLOW_INSECURE=1 is the
  explicit opt-out and warns on stderr.

- otelbridge/config.py:189 (CWE-201, redirect header forwarding) — FIXED. The
  OTLP/HTTP exporters now get a session that never follows redirects.
  `requests` strips Authorization only when the redirect changes host, so a
  3xx replayed x-api-key to the target and even kept Authorization on an
  https -> http downgrade to the same host.

- otelbridge/config.py:177 (endpoint precedence) — FIXED. `from_args_env` no
  longer pre-reads OTEL_EXPORTER_OTLP_ENDPOINT into an explicit exporter
  endpoint: doing so overrode the signal-specific _TRACES_ENDPOINT /
  _METRICS_ENDPOINT, inverting the OTLP spec order — the same defect already
  fixed for the header variables in c23b312. The SDK now resolves all three,
  and `effective_endpoint()` mirrors its order read-only for the startup line
  and the security check.

- otelbridge/bridge.py:207 (signal ownership) — FIXED. `loop.add_signal_handler`
  replaces rather than chains, and asyncio exposes no way to restore the
  displaced callback, so the bridge no longer touches a signal the running
  loop already handles; `install_signal_handlers=False` turns registration off
  entirely for embedders.

- otelbridge/mapper.py:611 (unbounded run.pipes) — FIXED. MAX_TRACKED_RUNS did
  not bound pipes: a run that keeps receiving events is re-inserted as
  most-recently-eventful and never evicted, while every begin (or event for a
  never-begun pipe) adds a _PipeState that only a matching end removes, and a
  reconnect does not replay missed ends. MAX_TRACKED_PIPES_PER_RUN now closes
  the least-recently-eventful pipe as unclosed, with recency updated on pipe
  activity rather than inferred from insertion order.

- otelbridge/setup.py:202 (tracer leak) — FIXED. A failure anywhere in the
  metric half left the BatchSpanProcessor worker running with no handle to
  stop it, because `shutdown` had not been created yet. That path now shuts
  the tracer provider down before re-raising.

Tests: 32 added (131 in tests/test_otel_*.py, was 99), covering stdout
redaction, the refusal and its opt-out, env-var and signal-specific endpoint
coverage, the non-redirecting session, the tracer teardown, signal ownership
and the pipe cap's LRU behaviour.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(sdk): skip the loop signal-handler ownership test on Windows

asyncio's ProactorEventLoop raises NotImplementedError from
add_signal_handler, so the test cannot set up its precondition there.
The bridge itself already tolerates that platform (the other signal
tests pass on Windows); only the test needs the platform guard.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(cli): document the otel --insecure flag in the site CLI reference

The `otel` command grew an `--insecure` opt-out for the cleartext-credential
guard, but the docs-site "Key flags" table was never updated to match. Users
hitting exit code 2 from `rocketride otel --endpoint http://tempo:4318
--headers Authorization=...` had no way to learn why from the CLI reference —
the flag was only documented in the client-python OTel bridge guide.

Add the missing row, naming the `ROCKETRIDE_OTEL_ALLOW_INSECURE` env var
alongside the flag so the site table matches otel-bridge.md and the argparse
help text.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(sdk): declare the otel test dependency and stop orphaning bridge providers

Two review findings on the `rocketride otel` bridge.

1. The opentelemetry dependency of the bridge tests was inherited, not
   declared. tests/test_otel_mapper.py and tests/test_otel_setup.py opened
   with pytest.importorskip('opentelemetry'), and nothing in the test path
   installs the 'otel' extra — the packages reached the engine interpreter
   only as a transitive dependency of the crewai node. Dropping or repinning
   that node would have turned ~1450 lines of mapper/setup coverage into a
   silent skip with CI still green.

   CI never pip-installs the package (pytest runs on the engine interpreter
   importing the package from the source checkout via tests/conftest.py), so the extras alone cannot fix this. The
   deps are added to packages/server/build-requirements.txt, which
   server:setup-test-deps installs through depends() under the merged
   constraints — the one install step that path goes through — and the
   'dev'/'test' extras now pull 'rocketride[otel]' so a source checkout
   matches. Both importorskip guards become hard imports: a missing
   dependency now fails collection instead of skipping.

2. build_providers() always built both provider halves and adopted its
   shutdown only when shutdown_fn was None, so an embedder passing its own
   shutdown_fn left a TracerProvider and its BatchSpanProcessor worker
   thread with nothing to stop them — and the half the caller had already
   supplied via a factory was built, wired and discarded.

   build_providers() takes with_tracer / with_meter (defaulting to the old
   behaviour) and run_bridge asks only for the half it is missing; a
   caller-supplied shutdown_fn is now chained in front of the providers'
   shutdown — caller first, providers in a finally — instead of replacing
   it. The CLI path passes neither argument and is unaffected.

Regression tests cover chain order, the provider shutdown surviving a
raising caller, and which half is built for each factory combination;
test_otel_bridge.py stays runnable without the extra. The embedded-use
ownership rules are documented in the bridge doc.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* build(client-python): install the otel test deps in the test step, not the engine

Review follow-up: the previous commit declared opentelemetry-sdk and
opentelemetry-exporter-otlp-proto-http in packages/server/build-requirements.txt.
That file is installed by server:setup-test-deps on every engine build, so every
engine install carried the OTel SDK and the OTLP exporter whether or not anyone
ran the bridge — which made the summary's "zero new base dependencies" true of
the package but not of the engine.

Move the cost to the step that needs it. A new client-python:setup-test-deps
action installs packages/client-python/tests/requirements.txt into the engine
interpreter through depends(), and runs only inside client-python:test, between
the parallel build group and the test-server bracket:

  server:build -> [nodes:build | ai:build | client-python:build]
               -> client-python:setup-test-deps
               -> py-test-server:setup -> client-python:run-pytest -> teardown

That ordering puts it after the node/ai requirement files land in the dist, so
depends() resolves against the complete constraint set, and before the bracket,
so a cold install does not hold the test server open. This mirrors the existing
packages/ai/tests/requirements.txt pattern (packages/ai/scripts/tasks.js).

packages/server/build-requirements.txt is restored byte-for-byte to its previous
state. The 'rocketride[otel]' self-reference in the dev/test extras stays — that
is the source-checkout path — and the hard imports in test_otel_mapper.py /
test_otel_setup.py stay, so a missing dependency still fails collection instead
of skipping. The tests no longer depend on the crewai node pulling opentelemetry
in transitively either way.

Docstrings in both test modules and the extras comment in pyproject.toml now
point at the new file and step.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(client-python): surface the OTel bridge on the Python SDK overview

Every other page under docs/public/python/ has a card in the SDK overview's
"Explore the SDK" grid; the newly re-homed otel-bridge page did not, so it
was reachable only from the sidebar and the CLI reference.

Adds the card last in the grid, matching sidebar_position: 13. Uses LuRadar,
which is not already taken by another card in this file and is present in the
Lucide set react-icons v5 ships.

This replaces the "OpenTelemetry export (`rocketride otel`)" section that used
to live in docs/README-python-client.md and packages/client-python/docs/index.md
— both deleted by the docs revamp (rocketride-org#2139).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(clients): declare otel-bridge as a Python-only site page

The client-docs gate requires both folders to publish the same page set, with
single-language extras declared in two places: PY_ONLY_PAGES/TS_ONLY_PAGES in
scripts/validate-client-docs.py, and the Site page set section of the schema.
docs/public/python/otel-bridge.md was added without either, so 'Docs site build'
failed with: python: undeclared site page otel-bridge.md.

The page is genuinely Python-only — the TypeScript SDK ships no OTel bridge
(no otel surface under packages/client-typescript/src, no page in
docs/public/typescript/) — so it belongs in PY_ONLY_PAGES rather than being
mirrored. Declared at sidebar_position 13, matching the TypeScript-only
database-sequelize extra.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(sdk): name the private SDK attribute the OTel bridge hooks

The bridge receives monitor events by assigning its dispatcher to
RocketRideClient._caller_on_event — the attribute EventMixin.__init__
creates from the public on_event= kwarg and EventMixin.on_event awaits
for every DAP envelope. It assigns rather than passing on_event= because
it is handed an already-built client and must restore the previous
handler on exit.

Nothing in the SDK's public surface pins that name, and a rename would
fail silently: assigning an unknown attribute to a Python object
succeeds, so the bridge would keep connecting, keep subscribing, keep
logging "reconnected", and never export another span.

Lift the string into CLIENT_EVENT_HOOK with the rationale next to it, so
the coupling is a single named seam a test can assert on instead of a
literal repeated across three call sites. No behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(sdk): pin the OTel bridge against the real RocketRideClient

The otel suite was mock-only past the mapper: test_otel_bridge.py and
test_otel_cli.py each define their own FakeClient, and the provider tests
monkeypatch build_providers, FlowSpanMapper and MetricsMapper out. That
covers the run loop's branches but leaves the bridge's one point of
coupling to the SDK untested — and a fake that defines
_caller_on_event itself can never notice the SDK dropping it.

Measured: renaming _caller_on_event to _caller_on_event_renamed across
mixins/events.py and mixins/data.py — a change that stops the bridge
exporting anything at all — leaves all 141 existing otel tests green.

Add tests/test_otel_client_contract.py, which constructs a REAL
RocketRideClient and stubs only the transport (call/is_connected); the
SDK's add_monitor bookkeeping, EventMixin.on_event dispatch, the mappers
and the OpenTelemetry provider pipeline all run for real:

  - three tests pinning CLIENT_EVENT_HOOK from different angles: the
    attribute exists in vars() on a real client, it is where the public
    on_event= kwarg lands, and the SDK's dispatch actually awaits it
  - run_bridge end to end on a real client, with events pushed through
    EventMixin.on_event rather than a fake's emit(), asserting routing,
    the unconsumed-event case, and hook restoration on shutdown
  - the wildcard subscription as the real SDK encodes it on the wire,
    plus the _monitor_keys entry the bridge relies on for replay after a
    reconnect
  - the 24-record wire fixture replayed to real exported spans, matching
    the 10-span forest test_otel_mapper.py gets by feeding the mappers
    directly, so the bridge provably adds, drops and reorders nothing

Under the same simulated rename, six of these seven fail.

Also add an opt-in live export against a real OTLP collector, gated on
ROCKETRIDE_OTEL_TEST_ENDPOINT via the same skipif shape
RocketRideClient_test.py uses for requires_llm. It starts no collector
and changes no CI step; without the env var it simply skips.

Note the new file in the client-python:setup-test-deps comment (it
imports the OTel SDK at collection time) and cross-reference it from
test_otel_bridge.py's docstring.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: nihalnihalani <24360630+nihalnihalani@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

builder Node builder tooling and ./builder workflows docs Documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants