Repository navigation
feat(run-logging): task-event continuum, DVR sessions, permanent trace identity - #1661
Conversation
… seq on every task event
Foundation phase of the run-logging plan (per-task JSONL event continuum).
Every task event now carries two header-level fields, stamped server-side
exactly once:
- eventTime (epoch seconds, float): stamped at the stdio ingress where raw
engine stdout frames become JSON events (TaskDbgStdio.on_event), so it
reflects emission time, not forwarding time. Derived events (apaevt_flow)
inherit their source trace event's time. An engine-provided stamp is never
overwritten, leaving the door open to in-engine stamping later.
- seq: a per-task continuum sequence seeded from the epoch in MICROSECONDS.
Why epoch-us: a new process's seed always exceeds every seq a previous run
of the same task identity issued (nothing sustains 1M events/sec), which
makes seq monotonic across runs, engine restarts, and supervisor crashes
with no persisted counter handshake; ~1.78e15 stays under JS
Number.MAX_SAFE_INTEGER. Legacy small per-endpoint DAP counters (from
build_event) are re-stamped; values at/above CONST_LOG_SEQ_FLOOR (1e14)
are recognized as continuum seqs and left untouched (idempotent).
Request/response seq is never rewritten — DAP correlates responses via
request_seq. raise_log_seq_floor() lets the L2 run-log writer lift the
seed to max(seed, control.lastSeq + 1) as belt-and-suspenders.
_forward_task_event stamps idempotently as a safety net so every broadcast
path (status updates, SSE, output) is covered by the same single counter.
Transport: plain output lines now preserve their originating channel as the
output category ('stderr' vs 'console') so the run log and Log page can
distinguish engine diagnostics from console prints.
Client folds (shared-ui project module): TraceEvent carries the new required
headers via parseServerEvent, and useTraceState row timestamps are now
event-sourced (eventTime * 1000). Date.now() is ELIMINATED from the fold —
no wall-clock fallback — so the same fold produces identical rows live and
on run-log replay (the L3+ player/DVR depend on this).
New logging constants (segment size, ring, chapters cap, payload caps,
history ages, sampling, read paging) land together in ai/constants.py and
are consumed progressively by later phases.
Part of the run-logging implementation plan (L1 of L1–L5).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…segments, control file, chapters, retention
One continuous log per task identity (projectId.source.runKind): runs are
chapters (lifecycle marker events) INSIDE the stream, never separate files —
which also resolves the deterministic-token collision (successive runs of
the same pipeline reuse the same identity, so per-run files were ambiguous
by construction).
RunLogWriter (new ai/modules/task/run_log.py):
- Append path: filtered event types (output, flow, errors, exit, lifecycle)
+ sampled status snapshots (one per CONST_LOG_STATUS_SAMPLE_SECONDS) +
per-event payload truncation that always preserves metadata/timestamps.
Synchronous and store-free — all store traffic is in a background worker.
- Sealing: 16 MB size seal on a line boundary (JSONL segments concatenate
byte-for-byte) plus a 24 h backstop seal so low-volume long-lived streams
cap host-loss tail and store lag at <= 1 object/day. Every store object is
written exactly once and never modified — no compaction, no multipart
visibility concerns, identical behavior on fs/S3/Azure/memory providers.
- Control file (./logs/{projectId}.{source}.{runKind}.json under the user's
store scope; token-free paths AND token-free log content): the time/seq ->
segment routing table and spooled/uploaded location ledger (segments carry
startTime/endTime/chapterStart/seq/id/state), stream startTime/endTime,
lastSeq, and the capped chapters list that renders the UI activity bar
from one small read.
- Retention: 64-segment ring + per-kind history age (dev 7d / deploy 30d);
eviction deletes store object AND spool copy (lease-deferred) and advances
the horizon, trimming chapters with it. Long-lived streams never finalize,
so the ring is their steady state.
- Concurrency disciplines: state flips to 'uploaded' BEFORE any spool
delete; SegmentLeases defers (never skips) deletes under readers; upload
failure leaves the spool copy serving reads and retries later.
- Recovery is STORE-SIDE ONLY (K8s container replacement kills the spool):
reopen trusts 'uploaded' entries, drops 'spooled' ones (the accepted loss
window), lifts the task's seq floor to lastSeq + 1 over the epoch-us seed
(belt and suspenders), and records a clock-anomaly lifecycle event in the
stream itself if the wall clock ever sits below the persisted lastSeq.
- Startup hygiene: sweep_spool_root() deletes stale spool dirs at module
init — spool state is never salvaged, and dev machines (whose filesystem,
unlike a replaced container, survives restarts) cannot leak disk.
Task wiring (task_engine.py): writer opens at subprocess start (run-begin
marker doubles as the chapter header; pipeline config hash, trigger, trace
level — never tokens), appends at the _forward_task_event choke point (what
clients see is exactly what replay reproduces), completes the chapter with
its outcome at termination, and records restarts as markers WITHOUT opening
a new chapter (a restart is not a new run). Logging is best-effort: any
failure leaves the task running unlogged.
Tests: 21 new unit tests over the filesystem provider (sealing, backstop,
upload transitions incl. failure/retry, two-run continuum + chapters, ring
and age eviction from both locations, chapters cap, store-side recovery,
clock-anomaly fallback, truncation, leases, spool sweep) + a conftest that
stubs the dist-only bootstrap modules (depends/rocketlib) so the suite runs
against the source tree; real modules always win when present. Full ai
suite: 1389 passed. test_task_engine's bare-Task helper seeds the two new
attributes _forward_task_event now consults.
Part of the run-logging implementation plan (L2 of L1–L5).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…inuum + SDK log namespaces Server: - RunLogReader (run_log.py): ranged/paged reads routed via the control file (seq/time/segment range forms), each segment resolved SPOOL-first with store fallback under read leases held only for the file read (a lazy generator would leave lease release to GC timing when a page limit stops mid-segment — deterministic release instead). Horizon-honest responses (truncatedAtSeq) and cursor paging (nextSeq). - The ACTIVE (unsealed) segment is now visible to readers: seal-or-continue keeps a completed run's tail (incl. its run-end marker) in the active spool file, so the control file carries an 'active' descriptor and the reader includes it as a virtual segment — the tail is readable the moment it is written. end_run closes the file HANDLE (no fd leak) but keeps the file + descriptor; the next run in-process reopens and continues it, restoring the descriptor so the shared segment keeps one coherent start time/seq. - Recovery unified under one rule — trust the filesystem: 'uploaded' entries always kept; 'spooled'/'active' entries kept (uploads re-queued) when their spool file exists (same-process reopen) and dropped when it died with a previous container (the accepted loss window). - WRITERS registry: a live stream's reader prefers the writer's in-memory control (fresher than the store copy by up to a seal interval), and delete routes through the writer's lock so it never races seal/upload. - delete: beforeTime (segment-granular — drop segments wholly older; trim chapters; advance horizon) or all (remove segments + control), both locations, lease/ordering disciplines. - cmd_log.py: LogCommands TaskConn mixin (subcommand table: chapters / read / delete) in the cmd_store/cmd_deploy pattern. Streams addressed by the plain identity tuple (projectId + source + runKind) — NEVER by token; store scoping derives from the AUTHENTICATED user only. task.monitor for reads, task.control for delete. SDKs (both, in sync): client.log namespace — chapters()/read()/delete() — with full type surfaces (LogChapter, LogActivitySpan, LogChaptersResult, LogEvent, LogReadResult, LogDeleteResult, LogRunKind) in TS and Python, plus the co-located guide page (docs/guide/methods/log.md) documenting the range forms, paging protocol, stamped headers, and permissions. Tests: 11 reader tests (chapters shape, ranged reads incl. types filter and time ranges, cursor paging terminates + matches unpaged, horizon truncation, spool-gone store fallback, delete beforeTime segment granularity + delete all, live-writer control preference). Full ai suite: 1400 passed. Part of the run-logging implementation plan (L3 server/SDK of L1–L5). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…er, PlayBar DVD transport, Log + Analyze projections useTaskEvents (modules/project/hooks) — THE single event-delivery point for one source's continuum, with media-player semantics per the plan: (1) DVR ring buffer: live events accumulate (capped, oldest dropped) even while paused or scrubbing — pause-live/scrub-back/GO LIVE need no refetch; (2) speed-aware prefetch: replay pages ahead of the playhead via the host-injected log.read pager, runway scaled by speed; (3) generation- cancelled seeks: in-flight fetches from a previous position are discarded on arrival; v1 contiguous-window policy (near seek reuses the buffer, far seek flushes + refills); (4) live/backfill merge deduped on seq; (5) pacing decoupled from fetching — dispatch at eventTime deltas / speed from the buffer. Also detects idle gaps between tracks and exposes the DVD controller (prev/next track, +/-30s skip, speeds, go-live); PLAY from a gap auto-skips to the next track's begin. Host-agnostic: readLog is injected, live events arrive via ingestLive — one instance per source section. PlayBar (components/play-bar) — the per-source transport + activity bar: |prev / rewind-30s / play-pause / forward-30s / next| + 1X/2X/4X/10X/25X + GO LIVE, over the single-lane timeline rendered PURELY from log.chapters metadata (no segment reads until a seek): filled spans per track (error tracks red, live span green, minimum span width keeps 3-second runs clickable), click-to-seek, cursor knob, retention-horizon axis label. LogPane — the human-readable projection of output/stderr/lifecycle events (the "Log page is a projection, not a second file" decision), with tail-pinning that detaches on manual scroll. AnalyzePane — efficiency analysis over the visible window from server-stamped emission times: per-component latency (reentrancy-safe enter/leave pairing), per-document durations, and idle gaps between runs — the trace-analyze concept, now on a real data substrate. run_log.py: the active segment opens line-buffered, so every appended event reaches the OS immediately and rrext_log reads of the ACTIVE segment are current to the last event — gap-free live composition (store + spool + active) without a separate in-memory tail structure (the L5 read-v2 mechanism, delivered by construction). Part of the run-logging implementation plan (L3 UI delivery + L4 player + L5 pieces of L1–L5). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ward The monitor forward (MonitorCommands.send_task_event) rebuilt each outgoing envelope with build_event(), which mints a fresh per-connection DAP seq and carries no eventTime — so live subscribers never saw the continuum stamps the run log records, even though every task event is stamped at stdio ingress. That silently broke everything that correlates live events with the logged stream: the client replay folds (which have no wall-clock fallback by design), the live/backfill dedupe seam in useTaskEvents, and the late-joiner DVR protocol (first live seq X -> read toSeq X-1). send_event() now takes optional event_time/seq kwargs that restore the original header stamps onto the built envelope (only as a pair, matching how Task.stamp_log_event issues them), and the monitor forward passes the stamps through from the source message. Request/response seq is untouched — only forwarded event messages are re-stamped, per the DAP request_seq correlation constraint. Extending send_event (rather than bypassing it to self.send) keeps one envelope-building path and the existing test seam. Part of the run-logging plan (L1 stamping, delivery to live monitors). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n unit + Design|Development|Deploy strip Restructures ProjectView to the final UI direction (v5, source-major), replacing the five page-level monitoring views (Status/Tokens/Flow/Trace/ Errors) that could only ever show the live task: - New SourceSection: the self-contained per-source unit. Each section carries its OWN pill bar (Status/Tokens/Flow/Trace/Errors/Log/Analyze), pane, and PlayBar bound to that source's run-log continuum. Live mode renders from host-fed state exactly as before; replay mode derives status from the last absolute snapshot in the visible window, folds trace/flow through the SAME parseServerEvent/useTraceState path used live, and projects Log/Analyze straight off the window. Seeking between tracks shows an explicit idle-gap state with previous/next-run actions (DVD semantics — PLAY from a gap auto-skips to the next track). - ProjectView page strip becomes DESIGN | DEVELOPMENT | DEPLOY. Development stacks dev-continuum sections; Deploy hosts an optional pipeline-level deployLifecycle slot (owned by the deploy feature) above deploy-continuum sections. Sources replay independently — no shared cursor. Save now lives ONLY on the Design page (inside the canvas); the monitoring pages carry no document actions. Persisted view modes from before the restructure migrate onto the new strip (old monitoring modes -> development). The Development entry carries the live error/warning count so problems stay glanceable from any page. - New host props: liveLogEvents (raw stamped envelopes; sections split them per source by body.source or the '<tokenhash8>.<source>' envelope id), readLog + fetchTimeline (stream-bound wrappers over client.log), and the deployLifecycle slot. traceEvents/onTraceClear are deprecated (kept so existing hosts compile) — sections fold trace state themselves, which is what makes replay identical to live. - VS Code webview host wired live-only for v1: accumulates this project's stamped events (capped, chunky trim — sections detect the shrink and re-feed their seq-deduped buffers) and passes them through; no replay bindings yet. The old trace accumulator and trace:clear plumbing are gone. Why source-major: each source is a separate stream/task, and coupling their players behind one page cursor (v3/v4 designs) made mixed states — one source live, one replaying — incoherent. One section per source is also exactly today's multi-status-card mental model, extended. Part of the run-logging plan (L3-L5 UI delivery). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rozen in-bundle, append-only Adding the SDK's new client.log namespace broke the shell-api contract: the frozen v0 bundle inlined a structural copy of RocketRideClient, and WorkspaceProvider's `client` prop put that copy in an input (contravariant) position — so ANY additive SDK member read as a breaking change, and shell:freeze could not mint a successor because its own pre-check compiles the same failing floor. Design-owner decision (Rod, 2026-07-21): the client is an MF shared singleton the shell serves to every app, so its API surface IS part of this contract — keep it inlined in the frozen bundle and make it append-only, and fix the root cause instead of exempting members. - WorkspaceProvider no longer takes client/isConnected props: it reads both from useShellConnection() (provider-less, ConnectionManager singleton — the exact source Shell was reading one level above and threading down). Its props are the named, documented IWorkspaceProviderProps. With this, the client appears in NO input position anywhere on the surface, which is what makes strict value floors compatible with additive SDK growth. Rule recorded in contract-hold.ts and freeze-shell-api.js: components source the client from useShellConnection()/getClient(), never accept it as a prop. - api.ts exports RocketRideClient as a named type. parseExportedTypes in the freeze script now also collects `export declare class` names (previously only interface/type — exported classes silently got no floor), so the client receives its own covariant per-version type floor: Frozen = Current — SDK additions pass, removing or narrowing anything ever frozen fails shell-ui's own tsc. - versions/v0.d.ts recreated (deliberate, review-visible deletion + fresh freeze): the new snapshot contains the full client including the log namespace, and the slimmed provider props. contract-check / index.ts / latest.ts regenerated; the floors remain their plain strict form for every member — no exemptions. Known limits, accepted for now (Rod): npm-release alignment still needs freeze-at-publish discipline; client member additions do not by themselves mint a new frozen version (name-set detector); method-parameter bivariance can pass some parameter narrowings. Verified: ./builder shell:check green, ./builder shell:freeze idempotent, full ./builder saas:build green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…verything, chapter self-heal
Large live-e2e iteration batch on the run-logging continuum. Every change
below came out of driving the feature against real runs; each area lists
what changed and why the previous behavior was wrong.
ENGINE / SUPERVISOR (packages/ai)
- Seq assignment happens exactly once per DELIVERED event. Task.build_event
now stamps the continuum headers itself (the base class's small
per-endpoint counter is replaced at build time), stdout ingress stamps
eventTime ONLY, and the derived apaevt_flow is built inside the
trace-level gate. Why: ingress used to burn a seq on messages that are
consumed without delivery (apaevt_trace becomes a derived flow event), so
every visible flow event consumed two seqs and the continuum was riddled
with gaps.
- Final zeroed status: _terminated's terminal broadcast IS the final status
(gauges zeroed, body.final=true), emitted after metrics stop and the
terminal-state assignment, with the run-log closed after it. Why: an
earlier final-zero was overwritten by a later teardown broadcast, leaving
charts holding a stale residual reading forever; the stream's last word
must be the zero. Outcome check reordered (CANCELLED before exitCode==0)
so user-stops log as 'cancelled', not 'ok'.
- Tool-call tracing: DataConn._tool wraps py_instance.invoke in
BEGIN/ENTER/LEAVE/END >DBG events via monitorOther('DBG', ...) — the same
monitor channel the C++ debugger uses — on the REAL pipe id the call runs
on (caller's open pipe or the borrowed pool pipe; no synthetic ids).
LEAVE carries result + durationMs + capped output, errors and
PreventDefault probes included. total_pipes is emitted as 0 and the
supervisor keeps its current count for such events. Why: control-plane
tool calls bypassed the pipeflow entirely and were invisible in
Trace/Flow/log/replay; the recorded durations immediately exposed a
21s-per-connection network stall that had been misattributed to the
transport.
- run_log records EVERY event type delivered to clients; the
LOGGED_EVENT_TYPES whitelist is deleted. Why: it is a LOG — replay and
post-hoc analysis need what clients actually saw, and a whitelist
silently drops any newly added event type. Status sampling (final-zero
exempt) and the per-event payload cap still bound volume.
- Dangling-chapter self-heal: opening a stream closes any chapter left with
endTime=null as outcome 'interrupted', ended at the stream's last
recorded activity (capped at the new run's begin). Why: killed/crashed
runs never write run-end; consumers treated every dangling chapter as
live (multiple LIVE entries in the run menu) and merged consecutive runs
into one track.
- Storage via the account FileStore (refuses an empty client_id) with
.logs/{projectId}/{source}.{runKind}.{seg:06d}.jsonl segments + per-stream
control file; flat $TEMP spool with pattern-anchored sweep; end_run
force-seals and uploads so the store holds the full run immediately.
Monitor forward passes eventTime/seq through so clients see the same
stamps the log recorded. Tests rewritten for the FileStore layout and the
record-everything append.
CLIENT DATA MODEL (shared-ui useTaskEvents)
- One DVR: a single seq-ordered buffer and a single position; 'live' is the
position pinned to the wall clock, replay is the same position moving at
speed; one transport ticker owns the clock. Why: live and replay were
separate code paths that diverged (frozen charts, jumped positions,
trailing dead space skipped on catch-up).
- Shaped reads: statusAt (last snapshot at-or-before position),
trackEvents (effective track begin -> min(position, end)), chartSeries
(1Hz grid merging tracks and dead zones), rangeEvents (Analyze brush),
and trackStats (Now/Avg/Peak/Min scoped to the effective track). Why for
trackStats: the chart is fixed at a one-minute window, so window-scoped
stats read zero whenever the run's activity fell outside the visible
minute; track scope answers what the run actually did.
- Late-joiner backfill: entering live with a log reader fetches the whole
open track (bounded fallback window until the chapters cache reveals the
begin, extending backward when it does), deduped on seq, draining
multi-page continuations on the pinned tick. Why: a view opened mid-run
had nothing from before its subscribe instant — status kept the chart
alive but Log/Trace/Flow stayed empty forever.
- Track resolution anchors the synthetic live track at the LATEST run-begin
marker in the buffer, and chapters are normalized so only the newest may
be open-ended. Why: with rapid dev restarts several runs die between
chapter-cache refreshes; anchoring at the first stray event merged them
all into one track (Trace showed every run combined).
- Seek reuse ladder (serve / extend-left / extend-forward / flush+refill)
with the anchor clamped to the stream floor. Why: every seek used to
flush and refetch, blanking all panes for a frame on a one-pixel nudge.
UI (shared-ui components + project module)
- Needle PlayBar: fixed needle, timeline slides underneath, wheel zoom,
thin-until-hover strip, grab-the-tape scrub, Shift-brush Analyze slice,
day-grouped run menu with real clock times; transport is
[Play|Pause] [speed] [|<] [<<] [>>] [>|] [readout] [GO LIVE], with all
replay-only controls (and GO LIVE itself) hidden while live. Why: the
previous bar could not navigate hundreds of runs across days, and live
mode showed controls that had no meaning at the head.
- Status pane: fixed one-minute chart (range toggle removed — wider spans
are the play bar's job), real clock axis labels ('Now' only when actually
live), stats footer fed by trackStats, 450px pane body with the footer
pinned and the chart centered.
- Trace: durations always render as seconds with two decimals (raw float
millisecond stamps reached the screen before); tool-lane renderers show
tool name/input and duration/output; scroll containment fixed (the pane
fills the body exactly and only the tree scrolls — previously two
scrollbars fought and tall traces were clipped by a non-flex wrapper).
- Log pane: Download Log (text projection) and Download Events (raw JSONL
of the run) actions; breathing room below the terminal box; empty state
replaces the frame instead of nesting inside it.
- Uniform EmptyState (the stock shared-ui placeholder) across
Tokens/Flow/Trace/Errors/Log/Analyze. Why: six panes had six different
ad-hoc empties; the style guide's control makes them one system. Flow's
Pipeline/Component toggle hides when there is no data to toggle over.
- Flow pane folds pipeflow.byPipe from the track's flow events at the
position. Why: status snapshots sample ~5s apart and documents finish in
under a second, so sampled byPipe was almost always the empty idle
instant.
Verified: full ai pytest suite green, tsc clean, saas + rocket-ui builds
green, live e2e against real runs (ingest, replay, seeks, tool calls,
multi-run restarts).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…cro-button tier
Stock components exist so every application looks and behaves the same — a
hand-rolled duplicate of a stock component is a defect. This lands the first
conversion of the stock-components adoption plan and removes the style tier
that triggered it.
- PipelineActions: the "Chat now" / "Endpoint Info" buttons are now stock
<Button small variant="primary|secondary">. The private micro-button tier
(btnSmBase + primaryBtn/secondaryBtn + the --rr-btn-sm-* sizing, plus the
unused endpoint-row styles) is DELETED. Why: it was a bespoke button
system living inside one component; if a smaller button size is ever
genuinely needed it must be a Button prop, never a per-component style
block. Clicks stop at the container so canvas node presses never bubble
into node selection/drag.
- StatusHeader: split into composable pieces — StatusHeaderInfo (state dot +
name + state + status subtitle) and StatusElapsed ("Started Xs ago") — so
Card-based hosts can place them in the Card's header/headerActions slots
instead of nesting a second header row. StatusHeader remains as the
self-contained composition for non-Card hosts. StatusActions now renders a
stock <Button small> (danger for Stop, primary for Run/Subscribe) per the
Card-header rule that header buttons are plain Button small.
- SourceSection: the section shell is now a real stock Card —
StatusHeaderInfo in the header slot, the action cluster + elapsed in
headerActions, the pill bar as the Card toolbar, and the pane body +
PlayBar as an edge-to-edge body (noBodyPadding). Why: the section was
spreading commonStyles.card and hand-building everything the Card
component already owns (header row, divider, actions cluster), which is
exactly how the Endpoint Info button ended up bespoke in the first place.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…pipeline settings Move "engine arguments" and "debug output" out of the per-connection-group local-engine settings and re-introduce them as per-task pipeline settings (taskArguments, pipelineDebugOutput) applied through `.use` at task start. Why: the old knobs were ENGINE PROCESS flags — changing tracing or adding an argument meant tearing down and respawning the local engine, and the settings were meaningless against a cloud connection (no local process to flag). But both knobs are really properties of a RUN, not of the engine: the engine already accepts per-task args via `.use`, and --trace=debugOut belongs to the task being debugged. Modeling them per-task makes them work identically on local, on-prem, and cloud connections, and takes effect on the next run instead of the next engine restart. Details: - config.ts: taskArguments + pipelineDebugOutput join the pipeline settings group (with getTaskArgs()); the local connection group keeps only engineVersion. The raw argument string is intentionally passed as a single entry — the engine shell-splits it, so quoted paths survive. - engine-local.ts: the engine now spawns with base args only; no per-task flags leak into the process command line. - debugger/session.ts: launch requests attach getTaskArgs() so F5 runs get the same per-task arguments as panel-started runs. - extension.ts: the engineArgs array->string migration and the old local.debugOutput/engineArgs key migrations are deleted with the keys themselves — there is nothing left to migrate into. - package.json/PipelineSettings/SettingsWebview/panels: the settings UI moves the two fields into the Pipeline section, adds enum descriptions and explicit ordering so the webview and the contributes.configuration model stay mirror images of the cloud app manifest. - shell-ui: SettingSchema documents that `enum` stays string[] per the frozen v0 contract while integer schemas (pipeline TTL) may carry numeric entries in manifest JSON at runtime — renderers go through String() and coerce back via `type`, which is what SettingsPage now does. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…actions use it The stock-Card migration replaced PipelineActions' private micro buttons with Button small (26px). That is correct in card headers but overflows the canvas source-node card: "Drop now" + "Endpoint Info" side by side are wider than the node. Per the rule that killed the micro tier in the first place — a smaller button size must be a Button prop, never a per-component style block — the old sizing (16px tall, 9px font, 0 6px padding, radius 3, weight 500) returns as a stock `mini` prop, and PipelineActions renders mini in both of its hosts. `mini` composes after `small` so it wins if both are set. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tity, live/replay parity
One coherent layer replaces the ad-hoc "subscribe + backfill" model: every
consumer of task events now reads from a LogEventStream SESSION over the
run-log continuum, and live delivery is just the last stop of the same walk.
== Client session (log-stream.ts / log_stream.py, both SDKs in lockstep) ==
- Live events are NOT a special case. Arrivals append to a "live bucket" that
is an ordinary CachedSegment with a sentinel id sorting after every server
segment, so history walks, trace collection, and status folds all traverse
disk segments and the live tail through one code path. INVARIANT: the live
bucket is never discarded — only prefixes proven to be server-backed leave
it. When the bucket exceeds 24MB the catalog is reloaded and the prefix
covered by sealed segments is dropped; extending the last disk segment
advances maxDiskSeq the same way. Why: any policy that could throw away a
live event that never reached disk silently loses data at exactly the
moment the user is watching it happen.
- Two delivery modes, no pump/poll machinery: HISTORY walks the cached
segments in order until exhausted, then flips to LIVE; in LIVE mode an
arrival appends to the bucket AND is delivered directly to the session
callback (when pinned + playing + past the watermark). The walk cursor
dedupes the overlap by seq. Why: the previous arrival-drop/poke iterations
had windows where events sat in the bucket with no deliverer — the
two-mode model has none, and disk state is irrelevant to live delivery.
- Segment cache is capped at 6 resident segments, evicting least-recently
USED (touched on ensure and on pump serve) — farthest-from-needle eviction
would evict exactly the segments a replay is about to need again.
- Sessions self-register their monitor (all event classes) on open and
deregister on close, refcounted. Why: consumers were silently missing
OUTPUT/TASK-class events because the host's canvas-oriented monitor
(summary/flow) was the only registration; the session knows what it needs
and must not depend on what some host happened to subscribe.
== Trace identity (contract) ==
- A trace IS its begin event's continuum seq. Slot ids recycle within a run
and fold docIds are synthetic per-client counters — neither survives a
refresh, both collide across runs. The begin seq is epoch-microsecond
seeded and globally unique for the life of the stream, so
getTrace(beginSeq) is deterministic from any position, live or replay,
in both SDKs: span-table binary search to the owning segment, then collect
the slot's flow events forward to its end, crossing segments and the live
bucket, with extend-on-miss for a still-active tail segment.
- SSE events ride along: the writer records which segments an in-flight
document's SSE touched, and trace collection includes apaevt_sse rows by
pipe_id, so node narration ("embedding chunk 3/10") appears inside the
call tree it belongs to.
== Server (task_engine.py / run_log.py) ==
- _forward_task_event stamps project_id + source into every task-scoped body
centrally, replacing per-event-builder stamping and envelope-id suffix
tricks. Why: clients must be able to filter membership with a pure body
test; identity established at ONE choke point cannot drift per event type.
- apaevt_task begin carries beginSeq — the run-begin lifecycle marker's seq
(the marker itself is log-only). Clients synthesize the chapter locally
from the announcement and join the server's chapter list by exact seq
equality. Why: a task-begin-triggered chapter refetch is both a race (the
control file may not be rewritten yet) and a thundering herd (every
watching client fetching at once); carrying the identity in the
announcement makes the local fix-up exact and free.
- Teardown order: the run log closes LAST, after the final zeroed status and
the terminal task/lifecycle forwards. Why: the log is the record; closing
it before the terminal events made every recorded run end without its end.
== UI (shared-ui project module) ==
- useTaskEvents consumes a session instead of listening to host events:
fold with runway throttling, FOLD_CAP honesty, and DEAD ZONE = CLEARED —
outside any chapter, every pane reports empty/zero at ANY position, live
or replay. Why: stale panels in the gap between runs read as data from a
run that never happened.
- |< seeks to the current chapter's begin (or the previous chapter's end)
and >| to the chapter end (or next begin), with an epsilon so landing on a
boundary resolves into the intended track.
- SourceSection owns the session lifecycle, keeps the local timeline fixed
up from task begin/end announcements (server list reconciled by the slow
poll only), and hosts the trace DetailPanel: right-docked, width persisted,
Prev/Next over the request list, TraceDetail fetching by beginSeq.
- Trace pane becomes a pure request list; TraceDetail is a standalone
component (also opened from Analyze) rendering the full call tree with
Data|JSON|Raw stock ToggleGroup and interleaved SSE narration rows.
- AnalyzePane's slowest-documents rows open the trace popup instead of
seeking — inspecting WHY a document was slow is the point of the table,
and the begin seq each row already carries is the popup's exact key.
- LogPane caps its projection honestly (notices for both the display cap and
the fold window) and downloads via a full continuum walk, so the file is
complete even when the screen is not.
- PlayBar colors the live block from task begin/end announcements
(runActive) rather than timeline completion guesses, with a square live
edge.
== Tests ==
- test_log_stream.py: golden suite proving play==read equivalence, splice
and console/status exactness, getTrace by begin seq across recycled slots,
SSE inclusion, active-segment growth, direct live delivery, raced-fetch
delivery, the 24MB split, and the LRU cap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🤖 Internal: Discord sync markerAuto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThis change adds persistent per-task run-log storage, server log commands, Python and TypeScript replay clients, environment-oriented project monitoring, global pipeline settings, and updated shell, shared UI, and frozen API contracts. ChangesRun-log continuum and task integration
Client SDKs
Project monitoring UI
Shell and configuration contracts
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Implements per-task run-log continuum support end-to-end: the engine/server now stamps every task event with stable eventTime + monotonic seq, persists events as segmentized JSONL, serves them via a unified rrext_log command, and exposes DVR-style replay through updated TypeScript/Python SDKs and updated UI consumers.
Changes:
- Add server-side run-log stamping, persistence, and
rrext_logDAP routing (chapters/read/segment/delete) plus supporting tests. - Add SDK surfaces (
client.log.*, DVRLogEventStream) and internal segment codecs for TS/Python, with co-located docs. - Update shared UI + VS Code extension to consume stamped events, support replay-oriented panes (Log/Analyze), and move task args/debug output into pipeline settings.
Reviewed changes
Copilot reviewed 80 out of 81 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/shell-api/versions/v0.d.ts | Updates frozen shell-api v0 contract with new log types/API and WorkspaceProvider prop shape. |
| packages/shared-ui/src/modules/project/utils.ts | Adds continuum stamp fields (eventTime, seq) into parsed flow/trace events. |
| packages/shared-ui/src/modules/project/types.ts | Makes trace events carry required continuum headers; updates view-mode model. |
| packages/shared-ui/src/modules/project/index.tsx | Re-exports new DVR/session hooks and panes (Log/Analyze). |
| packages/shared-ui/src/modules/project/hooks/useTraceState.ts | Adds track reset key and uses server-stamped timestamps/trace identity (beginSeq). |
| packages/shared-ui/src/modules/project/components/LogPane.tsx | New log-text projection pane with capped display + full-download support. |
| packages/shared-ui/src/modules/project/components/AnalyzePane.tsx | New analysis pane computing latency/doc spans/idle gaps from stamped events. |
| packages/shared-ui/src/index.ts | Exports PlayBar component from shared UI package. |
| packages/shared-ui/src/components/trace/renderers/render_tool.tsx | Adds renderer for tool-call trace lane (input/output summaries + trees). |
| packages/shared-ui/src/components/trace/renderers/index.tsx | Registers the new tool lane renderer and summary behavior. |
| packages/shared-ui/src/components/trace/index.tsx | Exports TraceDetail alongside Trace/JsonTree. |
| packages/shared-ui/src/components/tokens/Tokens.tsx | Replaces bespoke empty UI with shared EmptyState component. |
| packages/shared-ui/src/components/status/StatusHeader.tsx | Splits header into composable pieces; switches actions to stock Button. |
| packages/shared-ui/src/components/status/Status.tsx | Refactors to pure renderer fed by DVR getters; fixes chart range to 1 minute. |
| packages/shared-ui/src/components/status/CompletionsChart.tsx | Labels/tooltips based on real timestamps; removes parent-driven stat calc. |
| packages/shared-ui/src/components/pipeline-actions/PipelineActions.tsx | Uses stock Button with new mini size; stops bubbling into canvas interactions. |
| packages/shared-ui/src/components/flow/Flow.tsx | Uses EmptyState for “no active …” cases. |
| packages/shared-ui/src/components/button/Button.tsx | Adds mini button size for canvas-node chrome. |
| packages/client-typescript/src/client/types/log.ts | Adds public TS types for run-log chapters/reads/segments + DVR session types. |
| packages/client-typescript/src/client/types/index.ts | Re-exports new log types from the types barrel. |
| packages/client-typescript/src/client/log.ts | Adds LogApi namespace (chapters/read/segment/delete + openEventStream). |
| packages/client-typescript/src/client/log-codec.ts | Adds internal segment codec/decoder for keyframe+delta reconstruction. |
| packages/client-typescript/src/client/index.ts | Exports LogEventStream as public API; keeps codec internal. |
| packages/client-typescript/src/client/client.ts | Adds client.log lazy namespace accessor. |
| packages/client-typescript/docs/guide/methods/log.md | Adds TS SDK guide documentation for run-log APIs and DVR sessions. |
| packages/client-python/src/rocketride/types/log.py | Adds Python TypedDicts/Literals for run-log API types. |
| packages/client-python/src/rocketride/types/init.py | Exports run-log types from the Python SDK types package. |
| packages/client-python/src/rocketride/log.py | Adds Python LogApi namespace (chapters/read/segment/delete + open_event_stream). |
| packages/client-python/src/rocketride/client.py | Adds client.log cached_property for the run-log namespace. |
| packages/client-python/src/rocketride/_log_codec.py | Adds Python reference segment codec (keyframes + shallow deltas). |
| packages/client-python/src/rocketride/init.py | Exports LogEventStream from the Python SDK top-level. |
| packages/ai/tests/ai/modules/task/test_task_engine.py | Updates task test scaffolding for project_id and run-log seq/writer fields. |
| packages/ai/tests/ai/modules/task/test_run_log_reader.py | Adds reader tests for chapters/read/segment/delete + retention/paging coordination. |
| packages/ai/tests/ai/modules/task/conftest.py | Adds standalone-run stubs for runtime-only modules for task tests. |
| packages/ai/src/ai/modules/task/task_engine.py | Implements stamping (eventTime/seq), writer lifecycle, final status semantics, and log append path. |
| packages/ai/src/ai/modules/task/task_conn.py | Registers LogCommands mixin for rrext_log. |
| packages/ai/src/ai/modules/task/commands/cmd_monitor.py | Preserves continuum stamps when forwarding monitor events to clients. |
| packages/ai/src/ai/modules/task/commands/cmd_log.py | Adds rrext_log command router (chapters/read/segment/delete) with permission checks. |
| packages/ai/src/ai/modules/task/init.py | Sweeps stale spool roots on startup. |
| packages/ai/src/ai/modules/data/data_conn.py | Emits tool-call traces via DBG monitor channel with payload capping. |
| packages/ai/src/ai/constants.py | Adds run-log constants for seq floor, segmenting, retention, paging, and keyframe bounds. |
| packages/ai/src/ai/common/dap/transport_stdio.py | Preserves stderr vs stdout category when generating output events. |
| packages/ai/src/ai/common/dap/dap_conn.py | Allows send_event to carry through eventTime + seq for forwarded task events. |
| apps/vscode/src/providers/WelcomeProvider.ts | Stops threading debugOutput/engineArgs in welcome settings payload. |
| apps/vscode/src/providers/views/Welcome/WelcomeWebview.tsx | Updates defaults for new pipeline settings (TTL/trace/args/debug). |
| apps/vscode/src/providers/views/Settings/SettingsWebview.tsx | Extends settings model with taskArguments + pipelineDebugOutput; updates defaults. |
| apps/vscode/src/providers/views/Settings/PipelineSettings.tsx | Updates pipeline settings UX (TTL dropdown + task args + debug output). |
| apps/vscode/src/providers/views/Project/ProjectWebview.tsx | Switches to collecting stamped live events with caps; routes membership by project/body or id suffix. |
| apps/vscode/src/providers/views/components/panels/OnPremPanel.tsx | Removes debug output as a connection option; treats it as pipeline setting. |
| apps/vscode/src/providers/views/components/panels/LocalPanel.tsx | Removes debug output/engine args from connection options; keeps engineVersion only. |
| apps/vscode/src/providers/views/components/ConnectionConfig.tsx | Updates panel wiring for removed connection-level debug/args. |
| apps/vscode/src/providers/SidebarProvider.ts | Passes task args via .use using new ConfigManager accessor. |
| apps/vscode/src/providers/SettingsProvider.ts | Includes new pipeline settings in provider state sent to webviews. |
| apps/vscode/src/providers/ProjectProvider.ts | Passes task args via .use for project runs/restarts. |
| apps/vscode/src/extension.ts | Removes migration logic for deprecated engineArgs/debugOutput keys. |
| apps/vscode/src/engine/local/engine-local.ts | Removes engine-process debug/args flags; keeps server process base args only. |
| apps/vscode/src/debugger/session.ts | Passes task args via debug adapter launch requests. |
| apps/vscode/src/config.ts | Replaces engineArgs/debugOutput with taskArguments/pipelineDebugOutput + new getTaskArgs(). |
| apps/vscode/package.json | Removes deprecated engine arg/debug settings; adds pipeline TTL enum + new task args/debug settings. |
| apps/vscode/docs/usage.md | Documents new pipeline execution defaults (TTL choices, task args, debug output). |
| apps/vscode/docs/installation.md | Adds settings docs for taskArguments and pipelineDebugOutput. |
| apps/vscode/docs/index.md | Updates settings reference to match new task argument/debug model. |
| apps/shell-ui/src/workspace/WorkspaceContext.tsx | Stops taking client/isConnected as props; sources connection from useShellConnection. |
| apps/shell-ui/src/workspace/types.ts | Clarifies enum typing/handling for runtime numeric enums. |
| apps/shell-ui/src/views/settings/SettingsPage.tsx | Improves enum default display and numeric enum coercion. |
| apps/shell-ui/src/contract-hold.ts | Documents contract constraints around keeping client out of input positions. |
| apps/shell-ui/src/contract-check.generated.ts | Adds explicit type floors for RocketRideClient and WorkspaceProvider props. |
| apps/shell-ui/src/components/layout/Shell.tsx | Removes prop-threading of client into WorkspaceProvider. |
| apps/shell-ui/src/api.ts | Exports WorkspaceProvider props type and RocketRideClient type as contract surface. |
| apps/shell-ui/scripts/freeze-shell-api.js | Ensures exported classes are floored; documents client contravariance constraint. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
apps/vscode/src/config.ts (1)
233-245: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy liftMigrate existing task argument and debug settings instead of silently dropping them.
Removing the old per-engine keys while reading only the new global keys makes existing configured arguments/debug output disappear after upgrade.
apps/vscode/src/config.ts#L233-L245: add a one-time legacy fallback/migration when the new settings are unset, with defined handling for conflicting development/deployment values.apps/vscode/package.json#L188-L197: retain legacy settings as deprecated schema entries until migration is complete.apps/vscode/docs/index.md#L63-L64: document the upgrade mapping.apps/vscode/docs/installation.md#L45-L46: document the upgrade mapping.apps/vscode/docs/usage.md#L56-L63: document the upgrade mapping and changed global scope.As per coding guidelines, VS Code extension surface changes must be documented in
apps/vscode/docs/.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/vscode/src/config.ts` around lines 233 - 245, Update apps/vscode/src/config.ts lines 233-245 in refreshConfig to migrate legacy per-engine task-argument and debug settings when the new global settings are unset, with explicit handling for conflicting development/deployment values. In apps/vscode/package.json lines 188-197, retain those legacy settings as deprecated schema entries. Document the upgrade mapping and changed global scope in apps/vscode/docs/index.md lines 63-64, apps/vscode/docs/installation.md lines 45-46, and apps/vscode/docs/usage.md lines 56-63.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/vscode/src/providers/SidebarProvider.ts`:
- Around line 531-536: Update the sidebar run call in SidebarProvider’s
client.use invocation to include the same pipelineTraceLevel and ttl defaults
used by editor launches. Reuse the existing configuration/default sources rather
than introducing new values, while preserving the current pipeline, source,
args, and name fields.
In `@apps/vscode/src/providers/views/Project/ProjectWebview.tsx`:
- Around line 170-185: Update ProjectProvider.handleEvent and its monitor
subscription to forward stamped task output and lifecycle events identified by
envelope ID, not only events with body.project_id or summary/flow types. Resolve
the envelope’s task/token source through the project-safe mapping, then post
matching events to the webview so ProjectWebview’s isOurs buffering receives the
complete live log and trace.
In `@packages/client-python/src/rocketride/log_stream.py`:
- Around line 601-618: Update the timed playback branch in the event delivery
loop using _base_pos and _next_event so the playback clock advances before
re-checking pacing after a sleep. Ensure the pending event is not repeatedly
returned with the same delay, while preserving watermark updates, callback
delivery, and speed-zero batching behavior.
In `@packages/client-typescript/docs/guide/methods/log.md`:
- Around line 42-44: Update the getTrace example to pass the trace’s beginSeq
rather than the reused pipe-slot id, and make the corresponding change to the
Python example’s ['id'] argument. Preserve the surrounding trace-drilling
examples and use the beginSeq field from the getTraces summary.
In `@packages/client-typescript/src/client/log-stream.ts`:
- Around line 201-218: Update seek() and the related play() flow to advance
pumpEpoch before any repositioning or awaited seek work, retiring any in-flight
pump before it can resume and mutate basePos/watermark or invoke callbacks.
Ensure play() does not await seek(pos) while the old epoch remains active, and
apply the same ordering to the additional seek/play paths identified in the diff
while preserving normal positioning behavior.
- Around line 525-539: Update the replay-exhaustion branch around nextEvent and
refreshTimeline so events buffered by ingestLive during the asynchronous
transition are drained to the callback before committing to live mode. Preserve
ordering and ensure the live bucket is cleared or otherwise marked consumed,
then set pinned and basePos only after the buffered events have been delivered;
retain pause behavior for completed streams.
In `@packages/shared-ui/src/modules/project/hooks/useTraceState.ts`:
- Around line 140-153: Update the dependency array of the useEffect containing
the resetKeyRef comparison to include resetKey alongside traceEvents and flush.
Preserve the existing reset logic and effect behavior; do not add an eslint
suppression.
---
Outside diff comments:
In `@apps/vscode/src/config.ts`:
- Around line 233-245: Update apps/vscode/src/config.ts lines 233-245 in
refreshConfig to migrate legacy per-engine task-argument and debug settings when
the new global settings are unset, with explicit handling for conflicting
development/deployment values. In apps/vscode/package.json lines 188-197, retain
those legacy settings as deprecated schema entries. Document the upgrade mapping
and changed global scope in apps/vscode/docs/index.md lines 63-64,
apps/vscode/docs/installation.md lines 45-46, and apps/vscode/docs/usage.md
lines 56-63.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e03f2db8-9e58-4d0a-bbb2-d69eb5891b65
⛔ Files ignored due to path filters (1)
apps/shell-ui/src/contract-check.generated.tsis excluded by!**/*.generated.*
📒 Files selected for processing (80)
apps/shell-ui/scripts/freeze-shell-api.jsapps/shell-ui/src/api.tsapps/shell-ui/src/components/layout/Shell.tsxapps/shell-ui/src/contract-hold.tsapps/shell-ui/src/views/settings/SettingsPage.tsxapps/shell-ui/src/workspace/WorkspaceContext.tsxapps/shell-ui/src/workspace/types.tsapps/vscode/docs/index.mdapps/vscode/docs/installation.mdapps/vscode/docs/usage.mdapps/vscode/package.jsonapps/vscode/src/config.tsapps/vscode/src/debugger/session.tsapps/vscode/src/engine/local/engine-local.tsapps/vscode/src/extension.tsapps/vscode/src/providers/ProjectProvider.tsapps/vscode/src/providers/SettingsProvider.tsapps/vscode/src/providers/SidebarProvider.tsapps/vscode/src/providers/WelcomeProvider.tsapps/vscode/src/providers/views/Project/ProjectWebview.tsxapps/vscode/src/providers/views/Settings/PipelineSettings.tsxapps/vscode/src/providers/views/Settings/SettingsWebview.tsxapps/vscode/src/providers/views/Welcome/WelcomeWebview.tsxapps/vscode/src/providers/views/components/ConnectionConfig.tsxapps/vscode/src/providers/views/components/panels/LocalPanel.tsxapps/vscode/src/providers/views/components/panels/OnPremPanel.tsxpackages/ai/src/ai/common/dap/dap_conn.pypackages/ai/src/ai/common/dap/transport_stdio.pypackages/ai/src/ai/constants.pypackages/ai/src/ai/modules/data/data_conn.pypackages/ai/src/ai/modules/task/__init__.pypackages/ai/src/ai/modules/task/commands/cmd_log.pypackages/ai/src/ai/modules/task/commands/cmd_monitor.pypackages/ai/src/ai/modules/task/run_log.pypackages/ai/src/ai/modules/task/task_conn.pypackages/ai/src/ai/modules/task/task_engine.pypackages/ai/tests/ai/modules/task/conftest.pypackages/ai/tests/ai/modules/task/test_log_stream.pypackages/ai/tests/ai/modules/task/test_run_log.pypackages/ai/tests/ai/modules/task/test_run_log_reader.pypackages/ai/tests/ai/modules/task/test_task_engine.pypackages/client-python/src/rocketride/__init__.pypackages/client-python/src/rocketride/_log_codec.pypackages/client-python/src/rocketride/client.pypackages/client-python/src/rocketride/log.pypackages/client-python/src/rocketride/log_stream.pypackages/client-python/src/rocketride/types/__init__.pypackages/client-python/src/rocketride/types/log.pypackages/client-typescript/docs/guide/methods/log.mdpackages/client-typescript/src/client/client.tspackages/client-typescript/src/client/index.tspackages/client-typescript/src/client/log-codec.tspackages/client-typescript/src/client/log-stream.tspackages/client-typescript/src/client/log.tspackages/client-typescript/src/client/types/index.tspackages/client-typescript/src/client/types/log.tspackages/shared-ui/src/components/button/Button.tsxpackages/shared-ui/src/components/flow/Flow.tsxpackages/shared-ui/src/components/pipeline-actions/PipelineActions.tsxpackages/shared-ui/src/components/play-bar/PlayBar.tsxpackages/shared-ui/src/components/status/CompletionsChart.tsxpackages/shared-ui/src/components/status/Status.tsxpackages/shared-ui/src/components/status/StatusHeader.tsxpackages/shared-ui/src/components/tokens/Tokens.tsxpackages/shared-ui/src/components/trace/Trace.tsxpackages/shared-ui/src/components/trace/TraceDetail.tsxpackages/shared-ui/src/components/trace/index.tsxpackages/shared-ui/src/components/trace/renderers/index.tsxpackages/shared-ui/src/components/trace/renderers/render_tool.tsxpackages/shared-ui/src/index.tspackages/shared-ui/src/modules/project/ProjectView.tsxpackages/shared-ui/src/modules/project/components/AnalyzePane.tsxpackages/shared-ui/src/modules/project/components/LogPane.tsxpackages/shared-ui/src/modules/project/components/SourceSection.tsxpackages/shared-ui/src/modules/project/hooks/useTaskEvents.tspackages/shared-ui/src/modules/project/hooks/useTraceState.tspackages/shared-ui/src/modules/project/index.tsxpackages/shared-ui/src/modules/project/types.tspackages/shared-ui/src/modules/project/utils.tspackages/shell-api/versions/v0.d.ts
💤 Files with no reviewable changes (2)
- apps/vscode/src/providers/WelcomeProvider.ts
- apps/vscode/src/extension.ts
… drain, epoch races, vscode event routing Fixes for the verified review findings on the run-logging PR. Each carries the why; two findings were rejected (reasons on the PR threads: the conftest stubs are session-scope shims by design, and migrating engine-process args into per-task args would inject semantically wrong flags). - log-stream.ts / log_stream.py: timed playback (speed > 0) stalled forever on the first inter-event gap — the pacing branch slept/rescheduled and then recomputed the SAME delay from an unchanged playback clock. The clock now advances past the wait (epoch-guarded in TS so a retired timer cannot touch the live clock). The review caught the Python copy; the TS setTimeout re-entry had the identical stall and is fixed to match. - log-stream.ts / log_stream.py: an event arriving during the history-walk's final awaits (nextEvent exhaustion -> refreshTimeline) was buffered into the live bucket but never delivered — ingestLive only delivers arrivals AFTER the pin flips, and the walk was already done. The pump now synchronously drains bucket events above the watermark right after pinning; no await can interleave an arrival mid-drain in either runtime. - log-stream.ts: seek() now retires any in-flight pump (pumpEpoch bump) before its repositioning awaits — stopTimer only kills a SCHEDULED re-entry, so a pump parked on a fetch could resume against the old epoch and mutate watermark/basePos or fire a stale callback interleaved with the new position. Pump post-await guards also re-check playing/closed, so a pause landing mid-fetch cannot leak one extra delivery. (Python is immune: its seek cancels the pump task outright.) - vscode ProjectProvider/ProjectWebview: the host monitor subscribed only summary/flow and forwarded only status/flow events, so the webview's live log and run-active timeline never received output/lifecycle events. The monitor now adds output+task, handleEvent forwards every stamped event for the editor's project, and the webview's membership check becomes the pure body.project_id test (the envelope source-suffix hack is deleted — the server's central identity stamp made it obsolete). - vscode SidebarProvider: sidebar-started runs now pass the same pipelineTraceLevel + ttl as editor launches — a pipeline must run identically regardless of where it is started. - shared-ui utils.ts: parseServerEvent skips flow events missing the continuum header stamps instead of emitting undefined eventTime/seq into the folds (NaN timestamps, unstable ordering). A skip, not a wall-clock fallback: the folds are event-time-sourced by design; against a pre-continuum server the trace pane is empty rather than corrupt. - useTraceState.ts: the fold-reset effect reads resetKey but did not list it — the reset only fired because both callers change the traceEvents reference alongside it. The dep is added; the ref compare keeps re-runs idempotent. - docs guide/methods/log.md: both getTrace examples passed the recycled pipe slot (.id) instead of the trace identity (beginSeq), contradicting the contract stated in the same file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Review findings addressed in 41334c9 — all nine inline threads are replied to and resolved (7 fixed, 2 declined with reasons on the threads). On the outside-diff finding (config.ts: migrate legacy Beyond the reported findings, one bonus fix rode along: the TypeScript pump had the same timed-playback stall CodeRabbit found in the Python copy (its Validation: full ai suite green, 🤖 Generated with Claude Code |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/shared-ui/src/modules/project/utils.ts`:
- Around line 53-58: Update the guard in the fold/event processing utility to
require finite numeric values for both msg.eventTime and msg.seq, using
Number.isFinite rather than typeof checks alone. Apply the same finite-value
validation in the ProjectWebview live-event gate, preserving the existing
behavior of returning or skipping invalid events.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: f6c1c9d6-fb89-47fb-b67f-6a9c2bed39af
📒 Files selected for processing (8)
apps/vscode/src/providers/ProjectProvider.tsapps/vscode/src/providers/SidebarProvider.tsapps/vscode/src/providers/views/Project/ProjectWebview.tsxpackages/client-python/src/rocketride/log_stream.pypackages/client-typescript/docs/guide/methods/log.mdpackages/client-typescript/src/client/log-stream.tspackages/shared-ui/src/modules/project/hooks/useTraceState.tspackages/shared-ui/src/modules/project/utils.ts
PR #1661 rewired every accumulating pane — Trace, Errors, Status, Tokens, Flow, Log, Analyze — to read through a DVR session obtained from the host's `openEventStream`, a prop no host in this repo binds. Hosts that have not bridged it across their transport (the VS Code webview is one) passed a null session, `useTaskEvents.ingestLive` dropped every event, and the panes rendered their empty states mid-run with no error anywhere. Fall back to an in-memory store over the live events the host already delivers. It satisfies the same session contract `deliver` expects — a pure append, each event handed over exactly once, oldest first, across repeated `play` calls — and is keyed per stream identity so the several views `SourceSection` opens all share one buffer. Replay of recorded runs still requires the real stream. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(nodes): add tool_pipedrive CRM tool node
Exposes the Pipedrive REST API v1 to agents: 256 tools across 24 resource
groups (deals, persons, organizations, activities, pipelines, stages, notes,
leads, products, fields, files, users, roles, permission sets, teams, goals,
filters, webhooks, subscriptions, mailbox, call logs, projects, misc), plus a
generic `request` tool that reaches any remaining v1 endpoint through the same
auth, rate-limit and read-only layer.
Full coverage is more tools than an LLM can choose between reliably, so
`pipedrive.toolGroups` controls which groups are published. The default eight
core CRM groups publish 108 tools; `["all"]` publishes everything. Filtering
happens in `IInstance._collect_tool_methods()`, so a gated tool is invisible to
`tool.query` and refused by `tool.invoke` alike.
Client details specific to Pipedrive:
- `{"success", "data", "additional_data"}` envelope is unwrapped; `success:
false` on an HTTP 200 is raised as an error.
- `x-ratelimit-reset` is the seconds remaining in the window, not an epoch
timestamp as in the GitHub client this otherwise mirrors.
- Offset pagination (`start`/`limit`, max 500) everywhere except projects,
which use cursor pagination.
- Personal API tokens go in the `api_token` query parameter; JWT, long or
`Bearer `-prefixed values go in an Authorization header.
- Custom fields are grouped under `custom_fields` on read and written through
the `extra` passthrough that every create and update tool accepts.
Tests: 81 unit tests that need no credentials or network, plus an env-gated
live suite behind PIPEDRIVE_API_TOKEN, with record-creating tests additionally
behind PIPEDRIVE_ALLOW_WRITES.
Also corrects the stale node/service counts in docs/README-nodes.md
(88 -> 118 directories, 118 -> 148 services).
Closes #1675
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes): use the real Pipedrive brand mark for the tool_pipedrive icon
The first icon was a green uppercase "P" next to an unrelated dark dot. The
actual Pipedrive mark is a solid green disc carrying a white lowercase "p"
whose counter shows the disc through it.
Redrawn as plain geometry (disc r=128, stem 36x150, bowl r=56, counter r=27)
so it stays crisp at 16px with no gradients, clip paths or embedded raster.
Uses the default nonzero fill rule with the counter arc wound opposite to the
bowl. Under evenodd the stem and bowl would XOR where they overlap and punch a
green notch out of their junction; nonzero merges them solid while the opposing
winding still renders the counter as a hole.
Keeps the viewBox (SVGR runs with dimensions:false, so an icon without one
computes to zero width) and keeps two distinct colours, which makes the
auto-currentcolor svgo plugin treat it as a multicolour brand mark and pass the
authored colours through untinted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes): trace the tool_pipedrive icon from the supplied brand artwork
The previous version approximated the mark with circles. The real letterform
differs in three ways that are visible even at 16px: the stem carries a spur at
the top left, the bowl is an ellipse rather than a circle, and the counter is an
ellipse tangent to the stem's right edge.
Geometry was measured off the supplied 272x272 artwork rather than eyeballed —
connected-component analysis to isolate the counter, then a least-squares fit of
the bowl ellipse to the glyph's right-hand edge (max residual 2px). Rasterising
the traced primitives and diffing against the source leaves 1.47% of opaque
pixels differing, all of it a sub-pixel outline along the shape boundary.
Still a two-colour multicolour mark, so the auto-currentcolor svgo plugin passes
the authored colours through, and still nonzero winding with the counter wound
opposite the bowl so the stem and bowl merge solid while the counter stays open.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(nodes): warn when tool_pipedrive publishes too many tools for an agent
Full coverage is 256 tools. An operator can already reach that by selecting
groups, and nothing told them the agent's tool choice degrades long before it.
Adds a guard rail that counts published tools rather than groups, because group
sizes are lopsided: deals is 28 tools and permission_sets is 3, so a group count
bounds nothing useful. The threshold is 120, just under the 128-function cap
some providers impose on a single request and above the 107 the default groups
publish.
Crossing it warns and carries on. The tools are still published, in both
validateConfig (so it shows in the config panel while editing) and beginGlobal
(so it shows at pipeline start). Silently dropping tools an operator asked for
surfaces later as "the agent won't do X", which is worse than a noisy panel. The
warning is deliberately not raised from _collect_tool_methods, which runs on
every tool.query and tool.invoke.
"all" is exempt: it is already an explicit opt-in to the whole surface, and
scripted callers that never route through an LLM legitimately want it. Listing
every group by name is not exempt.
Group sizes are derived from the __pipedrive_group__ stamps rather than a
hand-maintained table, so a new tool cannot drift out of sync; a test asserts
every group in ALL_GROUPS has a count. Another test pins the advisory semantics
so the warning is not later "fixed" into a silent truncation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ai): flatten provider content blocks to visible text
Anthropic with extended thinking returns `content` as a list of typed
blocks, not a string. Every non-streaming path handed that list straight
to callers, who stringified it — so the answer arrived as the blocks'
repr, carrying the raw reasoning and its base64 signatures.
For agents it was worse than cosmetic: the ReAct parser reads output back
with a greedy `Action Input: (.*)`, so the serialized wrapper's trailing
`", "type": "text"}]` landed inside the tool arguments and every declared
field then reported as missing.
Extract the block vocabulary that only the streaming path knew into
`ai.common.utils.flatten_content_blocks` and route chat's invoke, the
stream fallback, and the agent's `extract_text` through it. The streaming
path now shares that vocabulary instead of duplicating it, so the paths
cannot drift; the fallback also forwards recovered reasoning to
`on_reasoning_chunk` rather than dropping it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes): make CrewAI tool calls and hierarchical kickoffs survive
Three defects made the CrewAI nodes fail on runs that should have worked.
Tool arguments were dropped. `_ToolInput` declared a single `input` field,
and CrewAI's argument filter intersects the model's emitted keys with the
schema's property names — so every real parameter was discarded before
validation, which then reported all required fields missing. Generate a
typed model per tool instead: JSON Schema types map to real annotations
(so pydantic coerces "123" -> 123), optional fields accept an explicit
null, and paramless tools get an empty schema. The JSON schema is no
longer appended to the description either; CrewAI renders args_schema into
the prompt itself, and the two copies disagreed.
`HostInvokeLLM` never declared `supports_function_calling()`. CrewAI
defines it on `LLM` but not on `BaseLLM`, and several call sites invoke it
unguarded — converter.py, tool_usage.py, reasoning_handler, and
task_evaluator. The AttributeError was swallowed by converter's broad
except and re-reported as "Failed to convert text into a Pydantic model".
Declare it False: the host channel is text-in / text-out and cannot honour
native tool calls, so every call belongs on the ReAct path. Pin crewai <2
for the same reason — 1.15.x moves the BaseLLM contract.
Hierarchical crews then died in planning. `Crew(planning=True)` runs a
task with `output_pydantic` set and drives it with the host wrapper, which
cannot promise JSON; unparseable plan text dead-ends the converter and
raises "Failed to get the Planning output". Make planning opt-in via a new
**Crew Planning** field, off by default.
Also clear `Task.tools` after construction: CrewAI back-fills them from
the task's agent, and a hierarchical crew resolves the executing agent's
toolset for the manager — which let the manager work a delegate's tools
itself instead of delegating.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes): move tool_pipedrive search to the v2 API
Pipedrive retired the v1 search routes: `/persons/search` and its siblings
answer 404 "Unknown method ." and `/itemSearch` answers a plain 404, so
every search tool the node published was dead. Non-search v1 endpoints
still route normally, so only the search calls move — a blanket swap would
break the many endpoints with no v2 equivalent.
Adds a v2 base URL (company-domain aware, sharing v1's normalisation) and
a v2 envelope wrapper: v2 replaced offset pagination with an opaque
`next_cursor`, which `paginated` would have missed entirely and reported
as a single page. Search tool schemas now advertise `cursor` instead of
`start`, and `item_search_by_field` follows v2's renames — `field_type` ->
`entity_type` with the "Field" suffix dropped from each value, and
`exact_match` replaced by the three-way `match` mode.
Also turns Tool groups into a multi-select dropdown with per-group tool
counts, so the field no longer requires typing group names from memory.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(shared-ui): keep live panes working without a host event stream
PR #1661 rewired every accumulating pane — Trace, Errors, Status, Tokens,
Flow, Log, Analyze — to read through a DVR session obtained from the
host's `openEventStream`, a prop no host in this repo binds. Hosts that
have not bridged it across their transport (the VS Code webview is one)
passed a null session, `useTaskEvents.ingestLive` dropped every event, and
the panes rendered their empty states mid-run with no error anywhere.
Fall back to an in-memory store over the live events the host already
delivers. It satisfies the same session contract `deliver` expects — a
pure append, each event handed over exactly once, oldest first, across
repeated `play` calls — and is keyed per stream identity so the several
views `SourceSection` opens all share one buffer. Replay of recorded runs
still requires the real stream.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(test): stop node packages being shadowed by same-named test dirs
nodes/test is a package whose subdirectories are named after node packages
(text_output/, response/, image_cleanup/, telegram/, tool_git/, ...). Two places
put that directory on sys.path so they could bare-import a test-root module:
conftest (for _sys_modules_guard) and cloud_tts/test_cloud_tts (for
test_contracts). Once nodes/test sits ahead of src/nodes, `import text_output`
resolves to the *test* package, whose __path__ holds no node module — so
`from text_output.instance import Instance` dies with ModuleNotFoundError.
Whether it bites was pure collection order. The affected modules bootstrap with
"insert src/nodes only if absent", which never re-prioritises: previously nothing
seeded src/nodes before cloud_tts (c) inserted nodes/test, so image_cleanup (i)
was the first to insert src/nodes and landed it in front. The new agent_crewai
tests seed src/nodes during collection of agent_crewai (a) instead, so cloud_tts
now stacks nodes/test on top of it and the guard short-circuits in all four later
modules — four collection errors, none of them in code this branch touches.
Import both test-root modules package-relative and drop the sys.path inserts, so
nodes/test never shadows a node package. cloud_tts gains the __init__.py the rest
of the node-named test dirs already have, which is what makes the relative import
resolvable. The four bootstraps now move src/nodes to the front unconditionally
rather than skipping when it is already somewhere on the path, and the guard suite
pins both invariants: nodes/test stays off sys.path, and a node package imported
by name resolves under src/nodes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes, shared-ui): address CodeRabbit review on the tool_pipedrive PR
Security
- pipedrive_client._company_host accepted anything up to the first slash, so a
configured domain of "evil.example#" built https://evil.example#.pipedrive.com
and Requests sent it — with the api_token or Authorization header attached — to
evil.example. Cut at the first path/query/fragment delimiter, refuse userinfo
and ports, lower-case, and require a single DNS label; anything else falls back
to the generic host.
- Free-form ids (lead uuids, permission-set ids, channel ids, goal ids, note
comment ids) were f-stringed into request paths unencoded, so a value carrying
"/", "..", "?" or "#" retargeted the call at another endpoint — for the DELETE
tools, a destructive one. Added _base.path_segment() and applied it at all 20
string-id sites.
- Pin requests>=2.34.2 to match constraints.lock, as tool_cognee/tool_github/
tool_n8n already do.
Correctness
- crewai_base._annotation_for passed prop['type'] straight to dict.get. JSON
Schema spells a nullable field as ["string", "null"], and an unhashable key
raises TypeError instead of returning the default — one such property aborted
the whole _build_crew_tools loop. Non-str types now fall back to Any, which is
what the docstring already promised.
- deal_followers_users_list GET the same /deals/{id}/followers as
deal_followers_list and only differed by running follower rows through
clean_user, emitting a user-shaped object built from mismatched fields.
Dropped it; resolving followers to users needs a /users/{id} call per follower.
- add_time (deals, organizations, persons) and origin_id (leads) were in the
write-key tuples but not in the matching schema props, so body_from could never
populate them. Added the properties.
- product_variation_update and both project-plan updates passed the builtin dict
as the response cleaner: dict(None) raises TypeError on an empty 204 body.
Switched to passthrough, as every other write already uses.
- liveEventSession.getStatus scanned from the live edge, contradicting the
documented "as of the session position" and the caller, which seeks to an
anchor first — a repositioned view seeded a replayed run with the current
status. Scan back from the watermark.
Maintainability
- Eight bulk deletes each repeated the ids guard, the CSV build and a trailing
_require_write(), and left ids optional so a missing argument surfaced as a
hand-rolled ValueError. Added _base._delete_bulk (write gate first), marked ids
required in every schema, and delegated all eight.
- projects._cursor_list re-implemented paging_params_v2 and the v2 envelope
shaping; it now uses the shared helpers, so the cursor contract has one
implementation instead of two that can drift.
- file_list advertised the shared 1-500 range, but /files documents a maximum
limit of 100. PAGING/paging_params/_list take a max_limit override and file_list
passes 100.
- _build_crew_tools' docstring still claimed the JSON Schema is embedded in the
tool description, which stopped being true when args_schema took over.
- Tests: 'ACME'.lower() evaluated to 'acme' before the call, so the case path was
never exercised; test_tools imported pipedrive_client bare, giving the module a
second identity in a run that also collects test_pipedrive; a vacuous
b64encode assert, a missing empty-pipelines guard, and a hard-coded field_id
of 1 that fails rather than skips on a fresh sandbox.
Two findings were not applied. `extra` is not dropped by fields/misc — body_from
merges args['extra'] by default (tools/_base.py). And the @patch/parametrize
signatures are already correct: mock injects positionally innermost-first while
pytest passes parametrize values by keyword, so the suggested reorder would bind
the mocks to the parametrized names.
New tests cover the rejected domains, path-segment encoding, non-str schema
types, the write-gate ordering, and the watermark-bounded status.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes): harden the tool_pipedrive path and bulk-delete helpers
Two follow-ups from the second CodeRabbit pass, both on helpers added in the
previous commit.
_delete_bulk merged the caller's `extra` pass-through after assigning the
validated `ids`, so `deal_delete_bulk({'ids': [1], 'extra': {'ids': '2,3'}})`
deleted 2 and 3 instead of 1. Merge `extra` first and let the validated CSV win.
The hand-rolled version this replaced had the same ordering, so the flaw predates
the helper.
path_segment relied on quote() alone, but "." and ".." contain nothing quote
escapes — `/notes/{id}/comments/..` survived encoding intact and addressed the
parent resource. Reject those two segments instead of encoding them.
Tests cover both: a dot-segment id raises before any request is made, and a
pass-through "ids" no longer displaces the validated list while other extra
parameters still reach the query string.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes, ai): keep the Pipedrive token out of transport error text
The personal-token path puts the credential in the `api_token` query parameter,
so requests bakes it into the URL. A transport-level failure (DNS, refused
connection, TLS, timeout) raises a RequestException whose text carries that URL,
and `raise ValueError(f'Pipedrive request failed: {exc}') from exc` copied the
token verbatim into the error the agent framework surfaces and logs. The chained
cause held the same URL, so scrubbing the message alone would still have leaked
it through any printed traceback.
Build the message from the exception type plus method and path — all secret-free,
and the type still distinguishes ConnectTimeout from SSLError from ConnectionError
— and raise `from None`. Verified this was the only such site: the JSON-decode
path stringifies `resp.text[:200]` rather than the URL, `_raise_pipedrive_error`
builds from parsed response fields, and the tenacity wrapper uses `reraise=True`
so no RetryError repr is introduced.
Three tests pin it, and all three fail against the previous code: the token
appears in neither the message nor the formatted traceback (that half is what
pins `from None`), and the message still names the failure and the call.
Also:
- notes' `add_time` was in _NOTE_WRITE_KEYS but not _NOTE_WRITE_PROPS, so
body_from could never populate it — the same class fixed for deals,
organizations, persons and leads in 7b3cc53. A scan of every
*_WRITE_KEYS/*_WRITE_PROPS pair confirms notes was the last one.
- ChatBase._chat had no direct test. flatten_content_blocks and extract_text are
covered, but nothing pinned the wiring that routes the provider's content
through them, which is the contract every non-streaming chat caller depends on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The bridge was written against a monitor protocol that carried no event timestamps, so every span used the bridge's own clock at event arrival and the guide documented that as a limitation. Since rocketride-org#1661 the engine stamps `body.eventTime` (epoch seconds, set once at ingress by the run-log continuum) on every forwarded event, and derived flow events inherit their source event's stamp — so the limitation is no longer true and the documentation shipped with it was wrong against develop. Span start/end and span-event timestamps now come from `body.eventTime`, making durations engine-measured instead of bridge-observed: WebSocket transport latency and event-loop scheduling no longer inflate every component span. The fallback is exactly the previous behaviour. A body with no usable `eventTime` (an older engine, or a non-float/NaN/non-positive value) and every span closed outside an event — shutdown, seeded-snapshot reconciliation, tracked-run cap eviction — use `time.time_ns()`, the same clock the SDK would have used. Because a run can therefore mix both sources, each open span records its own start and a close is clamped to never precede it, so no span can report a negative duration. Metric points are unchanged: OpenTelemetry metric instruments take no explicit timestamp, so they remain arrival-timed. That is now stated rather than left implicit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…mented ingester protocol (zero engine changes) (#1612) * feat(sdk): rocketride.otelbridge — OpenTelemetry export over the documented ingester protocol (#1609) Maps the engine's monitor event stream to standard OTel telemetry with zero engine/server changes — the client-side refinement invited by tracking issue #616 after the engine-invasive #498 was archived. - mapper.py: stateful FlowSpanMapper reconstructs span trees from apaevt_flow — task root span, pipe span per pipe id, component child spans paired by component identity (per the observability doc's warning against stack-position pairing); trace.error -> span status ERROR + exception event; apaevt_sse -> span events; unclosed spans at end/shutdown flagged rocketride.span.unclosed=true; per-pipe state is bounded. MetricsMapper maps apaevt_status_update snapshots to OTel instruments (object counts, rates, cpu/gpu memory + peaks). - Semantic conventions per the July 2026 semantic-conventions-genai snapshot: gen_ai.provider.name (not the deprecated gen_ai.system), gen_ai.operation.name from the well-known list only, span names '{operation} {model}' style, INTERNAL span kind for in-process component executions. - Privacy by default: lane payloads / prompt / result content never reach attributes or events unless include_content is set (8KB truncation cap when it is); enforced by a sentinel-scan test across all exported attributes and events. - setup.py: TracerProvider/MeterProvider + OTLP http/protobuf exporters; ALL opentelemetry imports are lazy so the base package imports without the extra. 53 unit tests (InMemorySpanExporter/InMemoryMetricReader; importorskip so the base CI matrix stays green), including a full replay of the captured real-engine event fixture. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(cli): add 'rocketride otel' subcommand + optional [otel] extra (#1609) rocketride otel [--endpoint URL] [--protocol http|grpc] [--service-name] [--headers k=v,...] [--include-content] [--no-metrics] + the standard connection args. Subscribes via rrext_monitor with token '*' (the observability doc's recommended ingestion scope, types TASK|SUMMARY| FLOW|SSE), dispatches to the mappers, reconnects with capped backoff and resubscribes (subscriptions are per-connection and not durable), and flushes exporters on SIGINT/SIGTERM. Endpoint resolution honors the standard OTLP env contract: explicit --endpoint > OTEL_EXPORTER_OTLP_TRACES_ENDPOINT (verbatim) > OTEL_EXPORTER_OTLP_ENDPOINT (base URL + /v1/traces|/v1/metrics) > default. Packaging: opentelemetry-sdk + OTLP http exporter live behind an optional extra — the base install gains zero dependencies, and running the subcommand without the extra exits 2 with the exact "pip install 'rocketride[otel]'" hint before any connection attempt. 25 tests: config precedence matrix, dispatch/reconnect/resubscribe with a fake client, shutdown flush ordering, missing-extra exit path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: OpenTelemetry bridge guide with Jaeger, Langfuse, and Datadog recipes Setup guide covering the span model (task -> pipe -> component tree), the privacy default and --include-content, the FLOW-requires-trace-level callout, reconnect semantics, and copy-paste recipes with exact OTLP endpoints/auth for Jaeger all-in-one, Langfuse (/api/public/otel, Basic auth), and Datadog. Notes that Jaeger does not ingest OTLP metrics — pair it with --no-metrics or point metrics at a collector. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): handle partial OpenTelemetry installs — guard the HTTP exporter import CI exposed a real environment our matrix missed: opentelemetry-api/sdk present as another package's transitive dependency (tool_daytona pulls them in) while the OTLP HTTP exporter is absent. Two fixes: - setup.py: the http trace/metric exporter imports are now guarded like the grpc ones — a partial install raises OtelNotInstalledError with the pip install 'rocketride[otel]' hint (CLI exit 2) instead of a raw ModuleNotFoundError. Verified live in a reproduced partial venv. - test_otel_setup.py: importorskip targets the most specific module the tests exercise (opentelemetry.exporter.otlp.proto.http), so the module skips cleanly in CI's partial environment; plus a regression test that simulates the partial install via a None sys.modules entry and asserts the friendly error. Partial venv: 63 passed + 1 module skip; full extra: 79 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): OTel bridge review round — privacy-gate errors, run-id promotion, header precedence, shutdown ordering CodeRabbit review round on #1612, each finding verified before acting: - mapper: trace.error no longer bypasses the privacy gate — by default only the error signal exports (status ERROR, bare exception event, generic description); --include-content restores verbatim text, now subject to the same 8KB truncation strings previously skipped. The sentinel privacy test plants the sentinel in trace.error. - mapper: a run first seen via flow events (before the task 'running' snapshot) is now promoted from its project|source fallback key to the canonical run id when it arrives — previously a second run state was created (broken parentage) and the orphaned fallback state leaked permanently. Confirmed with a failing repro; guard test ensures two distinct canonical runs sharing (project, source) never merge. - config/setup: only --headers become explicit exporter headers; env headers are left to the SDK so OTEL_EXPORTER_OTLP_TRACES_HEADERS / _METRICS_HEADERS signal precedence works (it was silently defeated). Precedence matrix tests added. - setup: meter provider shutdown wrapped in finally so metrics flush even when tracer shutdown raises. - CLI: --trace-level none message now says flow tracing stays disabled; docs recipes pass collector auth via OTEL_EXPORTER_OTLP_HEADERS env instead of argv (shell-history hygiene); error-mapping doc wording matches the implementation (component leave events). Full extra: 90 passed. Partial install (api/sdk without exporter): 69 passed + clean skip. ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): absorb stranded fallback run state when canonical state precedes it Follow-on to the run-promotion fix: an SSE event creates the canonical run state WITHOUT registering the (project, source) alias (SSE bodies carry no project/source per the protocol), so an id-less flow for the same run could still open a parallel fallback-keyed state — and because the canonical state already existed, the promotion branch was skipped, stranding the fallback state (kept alive forever by snapshot reconciliation) with duplicate spans. _get_run now reconciles whenever an event carries both the canonical id and project/source: no canonical state → promote (unchanged); canonical state exists → _absorb_fallback_run migrates non-colliding pipes (open spans re-stamped with the canonical run id), closes colliding pipes as rocketride.span.unclosed=true, and drops the fallback key. Only fallback-format keys are touched, so distinct canonical runs never merge (guard test unchanged and green). Both new regression tests failed before the fix. Full extra: 92 passed; partial install: 71 passed + expected skip; ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): stamp OTel spans with the engine's body.eventTime The bridge was written against a monitor protocol that carried no event timestamps, so every span used the bridge's own clock at event arrival and the guide documented that as a limitation. Since #1661 the engine stamps `body.eventTime` (epoch seconds, set once at ingress by the run-log continuum) on every forwarded event, and derived flow events inherit their source event's stamp — so the limitation is no longer true and the documentation shipped with it was wrong against develop. Span start/end and span-event timestamps now come from `body.eventTime`, making durations engine-measured instead of bridge-observed: WebSocket transport latency and event-loop scheduling no longer inflate every component span. The fallback is exactly the previous behaviour. A body with no usable `eventTime` (an older engine, or a non-float/NaN/non-positive value) and every span closed outside an event — shutdown, seeded-snapshot reconciliation, tracked-run cap eviction — use `time.time_ns()`, the same clock the SDK would have used. Because a run can therefore mix both sources, each open span records its own start and a close is clamped to never precede it, so no span can report a negative duration. Metric points are unchanged: OpenTelemetry metric instruments take no explicit timestamp, so they remain arrival-timed. That is now stated rather than left implicit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(client-python): publish the OTel bridge guide and mirror its CLI entry Two co-location gaps (AGENTS.md:5-28) and one stale claim. The 369-line guide lived at docs/README-otel-bridge.md, and root docs/ is not in the docs-site glob (packages/docs/scripts/lib/gather.js:25), so the guide was GitHub-only. Moved to packages/client-python/docs/otel-bridge.md, the tree already mounted at develop/python (packages/client-python/scripts/ tasks.js:236) alongside analytics.md and log.md, so it is gathered to the site at /develop/python/otel-bridge. Its in-repo links become site routes to match the sibling pages; the three references to it were repointed. packages/client-python/docs/index.md is the site's Python page and mirrors the PyPI README's ## CLI block line for line, but had no `rocketride otel` row and none of the OpenTelemetry paragraph the README gained. Added both. The guide's ## Limitations still said span timestamps are arrival time because the wire carries none; the previous commit made spans engine-timed, so that bullet now describes what actually remains bridge-timed (spans closed outside an event, pre-continuum engines, and all metric points). The Python blocks in the guide are also reformatted: ruff formats Markdown code blocks as of 0.16, and CI's ruff-action is unpinned, so the file failed `ruff format --check` today at either path. Repo-wide check is clean again. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(changelog): record the rocketride otel bridge under [Unreleased] CHANGELOG.md [Unreleased] ### Added is the convention for user-visible additions and is kept alphabetical by scope; a new CLI command and a new optional extra qualify. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(client-python): name Grafana Tempo in the OTel backend recipes Issue #1609's acceptance list says "Jaeger/Tempo", and the guide covered Tempo only implicitly as "Grafana stacks". Tempo is the OTLP receiver a Grafana user actually points the bridge at, and it needs one non-obvious step (enable distributor.receivers.otlp) plus a separate metrics backend, so it is worth naming rather than leaving a reader to infer. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sdk): address CodeRabbit review round on the rebased head Nine threads on the rebased head, all in the OTel bridge. Dispositions: - docs/otel-bridge.md:151 (Sensitive Data Exposure, CWE-522) — FIXED. The Langfuse and LangSmith recipes now prompt for the keys (printf + read -rs, spelled identically in bash and zsh) instead of embedding placeholders that operators replace with live secrets in a command line, and point at a secret manager as the better source. The base64 pipeline is now `printf '%s' ... | base64 | tr -d '\r\n'`: real Langfuse keys make the joined string 85 characters, past GNU base64's 76-character wrap, and command substitution strips only trailing newlines, so the embedded break would have produced an invalid Authorization header. - cli/commands/otel.py:207 (Sensitive Data Exposure, CWE-532) — FIXED. The startup line ran `config.endpoint` straight to stdout; an endpoint can carry userinfo credentials or a signed query string. It is now always passed through `redact_endpoint()`, which drops userinfo, query and fragment while keeping scheme/host/port and the ingest path operators actually verify. - cli/main.py:466, otelbridge/config.py:177, otelbridge/setup.py:123 (Cleartext Transmission, CWE-319) — FIXED, once, in one place. `validate_transport_security()` runs before any exporter exists (in `build_providers`, and again in the CLI so the failure is exit code 2 rather than a traceback): a credential-looking header name from `--headers` or any of the three OTEL_EXPORTER_OTLP_*HEADERS variables, plus a non-loopback cleartext endpoint for either signal, aborts startup. Header VALUES are never read or echoed — only names reach the message. Loopback stays exempt for local collectors; `--insecure` / ROCKETRIDE_OTEL_ALLOW_INSECURE=1 is the explicit opt-out and warns on stderr. - otelbridge/config.py:189 (CWE-201, redirect header forwarding) — FIXED. The OTLP/HTTP exporters now get a session that never follows redirects. `requests` strips Authorization only when the redirect changes host, so a 3xx replayed x-api-key to the target and even kept Authorization on an https -> http downgrade to the same host. - otelbridge/config.py:177 (endpoint precedence) — FIXED. `from_args_env` no longer pre-reads OTEL_EXPORTER_OTLP_ENDPOINT into an explicit exporter endpoint: doing so overrode the signal-specific _TRACES_ENDPOINT / _METRICS_ENDPOINT, inverting the OTLP spec order — the same defect already fixed for the header variables in c23b312. The SDK now resolves all three, and `effective_endpoint()` mirrors its order read-only for the startup line and the security check. - otelbridge/bridge.py:207 (signal ownership) — FIXED. `loop.add_signal_handler` replaces rather than chains, and asyncio exposes no way to restore the displaced callback, so the bridge no longer touches a signal the running loop already handles; `install_signal_handlers=False` turns registration off entirely for embedders. - otelbridge/mapper.py:611 (unbounded run.pipes) — FIXED. MAX_TRACKED_RUNS did not bound pipes: a run that keeps receiving events is re-inserted as most-recently-eventful and never evicted, while every begin (or event for a never-begun pipe) adds a _PipeState that only a matching end removes, and a reconnect does not replay missed ends. MAX_TRACKED_PIPES_PER_RUN now closes the least-recently-eventful pipe as unclosed, with recency updated on pipe activity rather than inferred from insertion order. - otelbridge/setup.py:202 (tracer leak) — FIXED. A failure anywhere in the metric half left the BatchSpanProcessor worker running with no handle to stop it, because `shutdown` had not been created yet. That path now shuts the tracer provider down before re-raising. Tests: 32 added (131 in tests/test_otel_*.py, was 99), covering stdout redaction, the refusal and its opt-out, env-var and signal-specific endpoint coverage, the non-redirecting session, the tracer teardown, signal ownership and the pipe cap's LRU behaviour. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(sdk): skip the loop signal-handler ownership test on Windows asyncio's ProactorEventLoop raises NotImplementedError from add_signal_handler, so the test cannot set up its precondition there. The bridge itself already tolerates that platform (the other signal tests pass on Windows); only the test needs the platform guard. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(cli): document the otel --insecure flag in the site CLI reference The `otel` command grew an `--insecure` opt-out for the cleartext-credential guard, but the docs-site "Key flags" table was never updated to match. Users hitting exit code 2 from `rocketride otel --endpoint http://tempo:4318 --headers Authorization=...` had no way to learn why from the CLI reference — the flag was only documented in the client-python OTel bridge guide. Add the missing row, naming the `ROCKETRIDE_OTEL_ALLOW_INSECURE` env var alongside the flag so the site table matches otel-bridge.md and the argparse help text. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sdk): declare the otel test dependency and stop orphaning bridge providers Two review findings on the `rocketride otel` bridge. 1. The opentelemetry dependency of the bridge tests was inherited, not declared. tests/test_otel_mapper.py and tests/test_otel_setup.py opened with pytest.importorskip('opentelemetry'), and nothing in the test path installs the 'otel' extra — the packages reached the engine interpreter only as a transitive dependency of the crewai node. Dropping or repinning that node would have turned ~1450 lines of mapper/setup coverage into a silent skip with CI still green. CI never pip-installs the package (pytest runs on the engine interpreter importing the package from the source checkout via tests/conftest.py), so the extras alone cannot fix this. The deps are added to packages/server/build-requirements.txt, which server:setup-test-deps installs through depends() under the merged constraints — the one install step that path goes through — and the 'dev'/'test' extras now pull 'rocketride[otel]' so a source checkout matches. Both importorskip guards become hard imports: a missing dependency now fails collection instead of skipping. 2. build_providers() always built both provider halves and adopted its shutdown only when shutdown_fn was None, so an embedder passing its own shutdown_fn left a TracerProvider and its BatchSpanProcessor worker thread with nothing to stop them — and the half the caller had already supplied via a factory was built, wired and discarded. build_providers() takes with_tracer / with_meter (defaulting to the old behaviour) and run_bridge asks only for the half it is missing; a caller-supplied shutdown_fn is now chained in front of the providers' shutdown — caller first, providers in a finally — instead of replacing it. The CLI path passes neither argument and is unaffected. Regression tests cover chain order, the provider shutdown surviving a raising caller, and which half is built for each factory combination; test_otel_bridge.py stays runnable without the extra. The embedded-use ownership rules are documented in the bridge doc. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * build(client-python): install the otel test deps in the test step, not the engine Review follow-up: the previous commit declared opentelemetry-sdk and opentelemetry-exporter-otlp-proto-http in packages/server/build-requirements.txt. That file is installed by server:setup-test-deps on every engine build, so every engine install carried the OTel SDK and the OTLP exporter whether or not anyone ran the bridge — which made the summary's "zero new base dependencies" true of the package but not of the engine. Move the cost to the step that needs it. A new client-python:setup-test-deps action installs packages/client-python/tests/requirements.txt into the engine interpreter through depends(), and runs only inside client-python:test, between the parallel build group and the test-server bracket: server:build -> [nodes:build | ai:build | client-python:build] -> client-python:setup-test-deps -> py-test-server:setup -> client-python:run-pytest -> teardown That ordering puts it after the node/ai requirement files land in the dist, so depends() resolves against the complete constraint set, and before the bracket, so a cold install does not hold the test server open. This mirrors the existing packages/ai/tests/requirements.txt pattern (packages/ai/scripts/tasks.js). packages/server/build-requirements.txt is restored byte-for-byte to its previous state. The 'rocketride[otel]' self-reference in the dev/test extras stays — that is the source-checkout path — and the hard imports in test_otel_mapper.py / test_otel_setup.py stay, so a missing dependency still fails collection instead of skipping. The tests no longer depend on the crewai node pulling opentelemetry in transitively either way. Docstrings in both test modules and the extras comment in pyproject.toml now point at the new file and step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(client-python): surface the OTel bridge on the Python SDK overview Every other page under docs/public/python/ has a card in the SDK overview's "Explore the SDK" grid; the newly re-homed otel-bridge page did not, so it was reachable only from the sidebar and the CLI reference. Adds the card last in the grid, matching sidebar_position: 13. Uses LuRadar, which is not already taken by another card in this file and is present in the Lucide set react-icons v5 ships. This replaces the "OpenTelemetry export (`rocketride otel`)" section that used to live in docs/README-python-client.md and packages/client-python/docs/index.md — both deleted by the docs revamp (#2139). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(clients): declare otel-bridge as a Python-only site page The client-docs gate requires both folders to publish the same page set, with single-language extras declared in two places: PY_ONLY_PAGES/TS_ONLY_PAGES in scripts/validate-client-docs.py, and the Site page set section of the schema. docs/public/python/otel-bridge.md was added without either, so 'Docs site build' failed with: python: undeclared site page otel-bridge.md. The page is genuinely Python-only — the TypeScript SDK ships no OTel bridge (no otel surface under packages/client-typescript/src, no page in docs/public/typescript/) — so it belongs in PY_ONLY_PAGES rather than being mirrored. Declared at sidebar_position 13, matching the TypeScript-only database-sequelize extra. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(sdk): name the private SDK attribute the OTel bridge hooks The bridge receives monitor events by assigning its dispatcher to RocketRideClient._caller_on_event — the attribute EventMixin.__init__ creates from the public on_event= kwarg and EventMixin.on_event awaits for every DAP envelope. It assigns rather than passing on_event= because it is handed an already-built client and must restore the previous handler on exit. Nothing in the SDK's public surface pins that name, and a rename would fail silently: assigning an unknown attribute to a Python object succeeds, so the bridge would keep connecting, keep subscribing, keep logging "reconnected", and never export another span. Lift the string into CLIENT_EVENT_HOOK with the rationale next to it, so the coupling is a single named seam a test can assert on instead of a literal repeated across three call sites. No behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(sdk): pin the OTel bridge against the real RocketRideClient The otel suite was mock-only past the mapper: test_otel_bridge.py and test_otel_cli.py each define their own FakeClient, and the provider tests monkeypatch build_providers, FlowSpanMapper and MetricsMapper out. That covers the run loop's branches but leaves the bridge's one point of coupling to the SDK untested — and a fake that defines _caller_on_event itself can never notice the SDK dropping it. Measured: renaming _caller_on_event to _caller_on_event_renamed across mixins/events.py and mixins/data.py — a change that stops the bridge exporting anything at all — leaves all 141 existing otel tests green. Add tests/test_otel_client_contract.py, which constructs a REAL RocketRideClient and stubs only the transport (call/is_connected); the SDK's add_monitor bookkeeping, EventMixin.on_event dispatch, the mappers and the OpenTelemetry provider pipeline all run for real: - three tests pinning CLIENT_EVENT_HOOK from different angles: the attribute exists in vars() on a real client, it is where the public on_event= kwarg lands, and the SDK's dispatch actually awaits it - run_bridge end to end on a real client, with events pushed through EventMixin.on_event rather than a fake's emit(), asserting routing, the unconsumed-event case, and hook restoration on shutdown - the wildcard subscription as the real SDK encodes it on the wire, plus the _monitor_keys entry the bridge relies on for replay after a reconnect - the 24-record wire fixture replayed to real exported spans, matching the 10-span forest test_otel_mapper.py gets by feeding the mappers directly, so the bridge provably adds, drops and reorders nothing Under the same simulated rename, six of these seven fail. Also add an opt-in live export against a real OTLP collector, gated on ROCKETRIDE_OTEL_TEST_ENDPOINT via the same skipif shape RocketRideClient_test.py uses for requires_llm. It starts no collector and changes no CI step; without the env var it simply skips. Note the new file in the client-python:setup-test-deps comment (it imports the OTel SDK at collection time) and cross-reference it from test_otel_bridge.py's docstring. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: nihalnihalani <24360630+nihalnihalani@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…e identity (rocketride-org#1661) * feat(task): L1 run-log source stamping — header eventTime + continuum seq on every task event Foundation phase of the run-logging plan (per-task JSONL event continuum). Every task event now carries two header-level fields, stamped server-side exactly once: - eventTime (epoch seconds, float): stamped at the stdio ingress where raw engine stdout frames become JSON events (TaskDbgStdio.on_event), so it reflects emission time, not forwarding time. Derived events (apaevt_flow) inherit their source trace event's time. An engine-provided stamp is never overwritten, leaving the door open to in-engine stamping later. - seq: a per-task continuum sequence seeded from the epoch in MICROSECONDS. Why epoch-us: a new process's seed always exceeds every seq a previous run of the same task identity issued (nothing sustains 1M events/sec), which makes seq monotonic across runs, engine restarts, and supervisor crashes with no persisted counter handshake; ~1.78e15 stays under JS Number.MAX_SAFE_INTEGER. Legacy small per-endpoint DAP counters (from build_event) are re-stamped; values at/above CONST_LOG_SEQ_FLOOR (1e14) are recognized as continuum seqs and left untouched (idempotent). Request/response seq is never rewritten — DAP correlates responses via request_seq. raise_log_seq_floor() lets the L2 run-log writer lift the seed to max(seed, control.lastSeq + 1) as belt-and-suspenders. _forward_task_event stamps idempotently as a safety net so every broadcast path (status updates, SSE, output) is covered by the same single counter. Transport: plain output lines now preserve their originating channel as the output category ('stderr' vs 'console') so the run log and Log page can distinguish engine diagnostics from console prints. Client folds (shared-ui project module): TraceEvent carries the new required headers via parseServerEvent, and useTraceState row timestamps are now event-sourced (eventTime * 1000). Date.now() is ELIMINATED from the fold — no wall-clock fallback — so the same fold produces identical rows live and on run-log replay (the L3+ player/DVR depend on this). New logging constants (segment size, ring, chapters cap, payload caps, history ages, sampling, read paging) land together in ai/constants.py and are consumed progressively by later phases. Part of the run-logging implementation plan (L1 of L1–L5). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(task): L2 run-log writer — per-task JSONL continuum with sealed segments, control file, chapters, retention One continuous log per task identity (projectId.source.runKind): runs are chapters (lifecycle marker events) INSIDE the stream, never separate files — which also resolves the deterministic-token collision (successive runs of the same pipeline reuse the same identity, so per-run files were ambiguous by construction). RunLogWriter (new ai/modules/task/run_log.py): - Append path: filtered event types (output, flow, errors, exit, lifecycle) + sampled status snapshots (one per CONST_LOG_STATUS_SAMPLE_SECONDS) + per-event payload truncation that always preserves metadata/timestamps. Synchronous and store-free — all store traffic is in a background worker. - Sealing: 16 MB size seal on a line boundary (JSONL segments concatenate byte-for-byte) plus a 24 h backstop seal so low-volume long-lived streams cap host-loss tail and store lag at <= 1 object/day. Every store object is written exactly once and never modified — no compaction, no multipart visibility concerns, identical behavior on fs/S3/Azure/memory providers. - Control file (./logs/{projectId}.{source}.{runKind}.json under the user's store scope; token-free paths AND token-free log content): the time/seq -> segment routing table and spooled/uploaded location ledger (segments carry startTime/endTime/chapterStart/seq/id/state), stream startTime/endTime, lastSeq, and the capped chapters list that renders the UI activity bar from one small read. - Retention: 64-segment ring + per-kind history age (dev 7d / deploy 30d); eviction deletes store object AND spool copy (lease-deferred) and advances the horizon, trimming chapters with it. Long-lived streams never finalize, so the ring is their steady state. - Concurrency disciplines: state flips to 'uploaded' BEFORE any spool delete; SegmentLeases defers (never skips) deletes under readers; upload failure leaves the spool copy serving reads and retries later. - Recovery is STORE-SIDE ONLY (K8s container replacement kills the spool): reopen trusts 'uploaded' entries, drops 'spooled' ones (the accepted loss window), lifts the task's seq floor to lastSeq + 1 over the epoch-us seed (belt and suspenders), and records a clock-anomaly lifecycle event in the stream itself if the wall clock ever sits below the persisted lastSeq. - Startup hygiene: sweep_spool_root() deletes stale spool dirs at module init — spool state is never salvaged, and dev machines (whose filesystem, unlike a replaced container, survives restarts) cannot leak disk. Task wiring (task_engine.py): writer opens at subprocess start (run-begin marker doubles as the chapter header; pipeline config hash, trigger, trace level — never tokens), appends at the _forward_task_event choke point (what clients see is exactly what replay reproduces), completes the chapter with its outcome at termination, and records restarts as markers WITHOUT opening a new chapter (a restart is not a new run). Logging is best-effort: any failure leaves the task running unlogged. Tests: 21 new unit tests over the filesystem provider (sealing, backstop, upload transitions incl. failure/retry, two-run continuum + chapters, ring and age eviction from both locations, chapters cap, store-side recovery, clock-anomaly fallback, truncation, leases, spool sweep) + a conftest that stubs the dist-only bootstrap modules (depends/rocketlib) so the suite runs against the source tree; real modules always win when present. Full ai suite: 1389 passed. test_task_engine's bare-Task helper seeds the two new attributes _forward_task_event now consults. Part of the run-logging implementation plan (L2 of L1–L5). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(task,client): L3 rrext_log — one read path over the run-log continuum + SDK log namespaces Server: - RunLogReader (run_log.py): ranged/paged reads routed via the control file (seq/time/segment range forms), each segment resolved SPOOL-first with store fallback under read leases held only for the file read (a lazy generator would leave lease release to GC timing when a page limit stops mid-segment — deterministic release instead). Horizon-honest responses (truncatedAtSeq) and cursor paging (nextSeq). - The ACTIVE (unsealed) segment is now visible to readers: seal-or-continue keeps a completed run's tail (incl. its run-end marker) in the active spool file, so the control file carries an 'active' descriptor and the reader includes it as a virtual segment — the tail is readable the moment it is written. end_run closes the file HANDLE (no fd leak) but keeps the file + descriptor; the next run in-process reopens and continues it, restoring the descriptor so the shared segment keeps one coherent start time/seq. - Recovery unified under one rule — trust the filesystem: 'uploaded' entries always kept; 'spooled'/'active' entries kept (uploads re-queued) when their spool file exists (same-process reopen) and dropped when it died with a previous container (the accepted loss window). - WRITERS registry: a live stream's reader prefers the writer's in-memory control (fresher than the store copy by up to a seal interval), and delete routes through the writer's lock so it never races seal/upload. - delete: beforeTime (segment-granular — drop segments wholly older; trim chapters; advance horizon) or all (remove segments + control), both locations, lease/ordering disciplines. - cmd_log.py: LogCommands TaskConn mixin (subcommand table: chapters / read / delete) in the cmd_store/cmd_deploy pattern. Streams addressed by the plain identity tuple (projectId + source + runKind) — NEVER by token; store scoping derives from the AUTHENTICATED user only. task.monitor for reads, task.control for delete. SDKs (both, in sync): client.log namespace — chapters()/read()/delete() — with full type surfaces (LogChapter, LogActivitySpan, LogChaptersResult, LogEvent, LogReadResult, LogDeleteResult, LogRunKind) in TS and Python, plus the co-located guide page (docs/guide/methods/log.md) documenting the range forms, paging protocol, stamped headers, and permissions. Tests: 11 reader tests (chapters shape, ranged reads incl. types filter and time ranges, cursor paging terminates + matches unpaged, horizon truncation, spool-gone store fallback, delete beforeTime segment granularity + delete all, live-writer control preference). Full ai suite: 1400 passed. Part of the run-logging implementation plan (L3 server/SDK of L1–L5). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(shared-ui,task): L3–L5 delivery layer — useTaskEvents smart buffer, PlayBar DVD transport, Log + Analyze projections useTaskEvents (modules/project/hooks) — THE single event-delivery point for one source's continuum, with media-player semantics per the plan: (1) DVR ring buffer: live events accumulate (capped, oldest dropped) even while paused or scrubbing — pause-live/scrub-back/GO LIVE need no refetch; (2) speed-aware prefetch: replay pages ahead of the playhead via the host-injected log.read pager, runway scaled by speed; (3) generation- cancelled seeks: in-flight fetches from a previous position are discarded on arrival; v1 contiguous-window policy (near seek reuses the buffer, far seek flushes + refills); (4) live/backfill merge deduped on seq; (5) pacing decoupled from fetching — dispatch at eventTime deltas / speed from the buffer. Also detects idle gaps between tracks and exposes the DVD controller (prev/next track, +/-30s skip, speeds, go-live); PLAY from a gap auto-skips to the next track's begin. Host-agnostic: readLog is injected, live events arrive via ingestLive — one instance per source section. PlayBar (components/play-bar) — the per-source transport + activity bar: |prev / rewind-30s / play-pause / forward-30s / next| + 1X/2X/4X/10X/25X + GO LIVE, over the single-lane timeline rendered PURELY from log.chapters metadata (no segment reads until a seek): filled spans per track (error tracks red, live span green, minimum span width keeps 3-second runs clickable), click-to-seek, cursor knob, retention-horizon axis label. LogPane — the human-readable projection of output/stderr/lifecycle events (the "Log page is a projection, not a second file" decision), with tail-pinning that detaches on manual scroll. AnalyzePane — efficiency analysis over the visible window from server-stamped emission times: per-component latency (reentrancy-safe enter/leave pairing), per-document durations, and idle gaps between runs — the trace-analyze concept, now on a real data substrate. run_log.py: the active segment opens line-buffered, so every appended event reaches the OS immediately and rrext_log reads of the ACTIVE segment are current to the last event — gap-free live composition (store + spool + active) without a separate in-memory tail structure (the L5 read-v2 mechanism, delivered by construction). Part of the run-logging implementation plan (L3 UI delivery + L4 player + L5 pieces of L1–L5). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(task,dap): preserve continuum stamps across the monitor event forward The monitor forward (MonitorCommands.send_task_event) rebuilt each outgoing envelope with build_event(), which mints a fresh per-connection DAP seq and carries no eventTime — so live subscribers never saw the continuum stamps the run log records, even though every task event is stamped at stdio ingress. That silently broke everything that correlates live events with the logged stream: the client replay folds (which have no wall-clock fallback by design), the live/backfill dedupe seam in useTaskEvents, and the late-joiner DVR protocol (first live seq X -> read toSeq X-1). send_event() now takes optional event_time/seq kwargs that restore the original header stamps onto the built envelope (only as a pair, matching how Task.stamp_log_event issues them), and the monitor forward passes the stamps through from the source message. Request/response seq is untouched — only forwarded event messages are re-stamped, per the DAP request_seq correlation constraint. Extending send_event (rather than bypassing it to self.send) keeps one envelope-building path and the existing test seam. Part of the run-logging plan (L1 stamping, delivery to live monitors). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(shared-ui,vscode): source-major environment pages — SourceSection unit + Design|Development|Deploy strip Restructures ProjectView to the final UI direction (v5, source-major), replacing the five page-level monitoring views (Status/Tokens/Flow/Trace/ Errors) that could only ever show the live task: - New SourceSection: the self-contained per-source unit. Each section carries its OWN pill bar (Status/Tokens/Flow/Trace/Errors/Log/Analyze), pane, and PlayBar bound to that source's run-log continuum. Live mode renders from host-fed state exactly as before; replay mode derives status from the last absolute snapshot in the visible window, folds trace/flow through the SAME parseServerEvent/useTraceState path used live, and projects Log/Analyze straight off the window. Seeking between tracks shows an explicit idle-gap state with previous/next-run actions (DVD semantics — PLAY from a gap auto-skips to the next track). - ProjectView page strip becomes DESIGN | DEVELOPMENT | DEPLOY. Development stacks dev-continuum sections; Deploy hosts an optional pipeline-level deployLifecycle slot (owned by the deploy feature) above deploy-continuum sections. Sources replay independently — no shared cursor. Save now lives ONLY on the Design page (inside the canvas); the monitoring pages carry no document actions. Persisted view modes from before the restructure migrate onto the new strip (old monitoring modes -> development). The Development entry carries the live error/warning count so problems stay glanceable from any page. - New host props: liveLogEvents (raw stamped envelopes; sections split them per source by body.source or the '<tokenhash8>.<source>' envelope id), readLog + fetchTimeline (stream-bound wrappers over client.log), and the deployLifecycle slot. traceEvents/onTraceClear are deprecated (kept so existing hosts compile) — sections fold trace state themselves, which is what makes replay identical to live. - VS Code webview host wired live-only for v1: accumulates this project's stamped events (capped, chunky trim — sections detect the shrink and re-feed their seq-deduped buffers) and passes them through; no replay bindings yet. The old trace accumulator and trace:clear plumbing are gone. Why source-major: each source is a separate stream/task, and coupling their players behind one page cursor (v3/v4 designs) made mixed states — one source live, one replaying — incoherent. One section per source is also exactly today's multi-status-card mental model, extended. Part of the run-logging plan (L3-L5 UI delivery). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(shell-ui,shell-api): recreate shell-api v0 with the SDK client frozen in-bundle, append-only Adding the SDK's new client.log namespace broke the shell-api contract: the frozen v0 bundle inlined a structural copy of RocketRideClient, and WorkspaceProvider's `client` prop put that copy in an input (contravariant) position — so ANY additive SDK member read as a breaking change, and shell:freeze could not mint a successor because its own pre-check compiles the same failing floor. Design-owner decision (Rod, 2026-07-21): the client is an MF shared singleton the shell serves to every app, so its API surface IS part of this contract — keep it inlined in the frozen bundle and make it append-only, and fix the root cause instead of exempting members. - WorkspaceProvider no longer takes client/isConnected props: it reads both from useShellConnection() (provider-less, ConnectionManager singleton — the exact source Shell was reading one level above and threading down). Its props are the named, documented IWorkspaceProviderProps. With this, the client appears in NO input position anywhere on the surface, which is what makes strict value floors compatible with additive SDK growth. Rule recorded in contract-hold.ts and freeze-shell-api.js: components source the client from useShellConnection()/getClient(), never accept it as a prop. - api.ts exports RocketRideClient as a named type. parseExportedTypes in the freeze script now also collects `export declare class` names (previously only interface/type — exported classes silently got no floor), so the client receives its own covariant per-version type floor: Frozen = Current — SDK additions pass, removing or narrowing anything ever frozen fails shell-ui's own tsc. - versions/v0.d.ts recreated (deliberate, review-visible deletion + fresh freeze): the new snapshot contains the full client including the log namespace, and the slimmed provider props. contract-check / index.ts / latest.ts regenerated; the floors remain their plain strict form for every member — no exemptions. Known limits, accepted for now (Rod): npm-release alignment still needs freeze-at-publish discipline; client member additions do not by themselves mint a new frozen version (name-set detector); method-parameter bivariance can pass some parameter narrowings. Verified: ./builder shell:check green, ./builder shell:freeze idempotent, full ./builder saas:build green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(run-logging): unified DVR, one-seq stamping, tool tracing, log-everything, chapter self-heal Large live-e2e iteration batch on the run-logging continuum. Every change below came out of driving the feature against real runs; each area lists what changed and why the previous behavior was wrong. ENGINE / SUPERVISOR (packages/ai) - Seq assignment happens exactly once per DELIVERED event. Task.build_event now stamps the continuum headers itself (the base class's small per-endpoint counter is replaced at build time), stdout ingress stamps eventTime ONLY, and the derived apaevt_flow is built inside the trace-level gate. Why: ingress used to burn a seq on messages that are consumed without delivery (apaevt_trace becomes a derived flow event), so every visible flow event consumed two seqs and the continuum was riddled with gaps. - Final zeroed status: _terminated's terminal broadcast IS the final status (gauges zeroed, body.final=true), emitted after metrics stop and the terminal-state assignment, with the run-log closed after it. Why: an earlier final-zero was overwritten by a later teardown broadcast, leaving charts holding a stale residual reading forever; the stream's last word must be the zero. Outcome check reordered (CANCELLED before exitCode==0) so user-stops log as 'cancelled', not 'ok'. - Tool-call tracing: DataConn._tool wraps py_instance.invoke in BEGIN/ENTER/LEAVE/END >DBG events via monitorOther('DBG', ...) — the same monitor channel the C++ debugger uses — on the REAL pipe id the call runs on (caller's open pipe or the borrowed pool pipe; no synthetic ids). LEAVE carries result + durationMs + capped output, errors and PreventDefault probes included. total_pipes is emitted as 0 and the supervisor keeps its current count for such events. Why: control-plane tool calls bypassed the pipeflow entirely and were invisible in Trace/Flow/log/replay; the recorded durations immediately exposed a 21s-per-connection network stall that had been misattributed to the transport. - run_log records EVERY event type delivered to clients; the LOGGED_EVENT_TYPES whitelist is deleted. Why: it is a LOG — replay and post-hoc analysis need what clients actually saw, and a whitelist silently drops any newly added event type. Status sampling (final-zero exempt) and the per-event payload cap still bound volume. - Dangling-chapter self-heal: opening a stream closes any chapter left with endTime=null as outcome 'interrupted', ended at the stream's last recorded activity (capped at the new run's begin). Why: killed/crashed runs never write run-end; consumers treated every dangling chapter as live (multiple LIVE entries in the run menu) and merged consecutive runs into one track. - Storage via the account FileStore (refuses an empty client_id) with .logs/{projectId}/{source}.{runKind}.{seg:06d}.jsonl segments + per-stream control file; flat $TEMP spool with pattern-anchored sweep; end_run force-seals and uploads so the store holds the full run immediately. Monitor forward passes eventTime/seq through so clients see the same stamps the log recorded. Tests rewritten for the FileStore layout and the record-everything append. CLIENT DATA MODEL (shared-ui useTaskEvents) - One DVR: a single seq-ordered buffer and a single position; 'live' is the position pinned to the wall clock, replay is the same position moving at speed; one transport ticker owns the clock. Why: live and replay were separate code paths that diverged (frozen charts, jumped positions, trailing dead space skipped on catch-up). - Shaped reads: statusAt (last snapshot at-or-before position), trackEvents (effective track begin -> min(position, end)), chartSeries (1Hz grid merging tracks and dead zones), rangeEvents (Analyze brush), and trackStats (Now/Avg/Peak/Min scoped to the effective track). Why for trackStats: the chart is fixed at a one-minute window, so window-scoped stats read zero whenever the run's activity fell outside the visible minute; track scope answers what the run actually did. - Late-joiner backfill: entering live with a log reader fetches the whole open track (bounded fallback window until the chapters cache reveals the begin, extending backward when it does), deduped on seq, draining multi-page continuations on the pinned tick. Why: a view opened mid-run had nothing from before its subscribe instant — status kept the chart alive but Log/Trace/Flow stayed empty forever. - Track resolution anchors the synthetic live track at the LATEST run-begin marker in the buffer, and chapters are normalized so only the newest may be open-ended. Why: with rapid dev restarts several runs die between chapter-cache refreshes; anchoring at the first stray event merged them all into one track (Trace showed every run combined). - Seek reuse ladder (serve / extend-left / extend-forward / flush+refill) with the anchor clamped to the stream floor. Why: every seek used to flush and refetch, blanking all panes for a frame on a one-pixel nudge. UI (shared-ui components + project module) - Needle PlayBar: fixed needle, timeline slides underneath, wheel zoom, thin-until-hover strip, grab-the-tape scrub, Shift-brush Analyze slice, day-grouped run menu with real clock times; transport is [Play|Pause] [speed] [|<] [<<] [>>] [>|] [readout] [GO LIVE], with all replay-only controls (and GO LIVE itself) hidden while live. Why: the previous bar could not navigate hundreds of runs across days, and live mode showed controls that had no meaning at the head. - Status pane: fixed one-minute chart (range toggle removed — wider spans are the play bar's job), real clock axis labels ('Now' only when actually live), stats footer fed by trackStats, 450px pane body with the footer pinned and the chart centered. - Trace: durations always render as seconds with two decimals (raw float millisecond stamps reached the screen before); tool-lane renderers show tool name/input and duration/output; scroll containment fixed (the pane fills the body exactly and only the tree scrolls — previously two scrollbars fought and tall traces were clipped by a non-flex wrapper). - Log pane: Download Log (text projection) and Download Events (raw JSONL of the run) actions; breathing room below the terminal box; empty state replaces the frame instead of nesting inside it. - Uniform EmptyState (the stock shared-ui placeholder) across Tokens/Flow/Trace/Errors/Log/Analyze. Why: six panes had six different ad-hoc empties; the style guide's control makes them one system. Flow's Pipeline/Component toggle hides when there is no data to toggle over. - Flow pane folds pipeflow.byPipe from the track's flow events at the position. Why: status snapshots sample ~5s apart and documents finish in under a second, so sampled byPipe was almost always the empty idle instant. Verified: full ai pytest suite green, tsc clean, saas + rocket-ui builds green, live e2e against real runs (ingest, replay, seeks, tool calls, multi-run restarts). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(shared-ui): SourceSection on the stock Card; kill the bespoke micro-button tier Stock components exist so every application looks and behaves the same — a hand-rolled duplicate of a stock component is a defect. This lands the first conversion of the stock-components adoption plan and removes the style tier that triggered it. - PipelineActions: the "Chat now" / "Endpoint Info" buttons are now stock <Button small variant="primary|secondary">. The private micro-button tier (btnSmBase + primaryBtn/secondaryBtn + the --rr-btn-sm-* sizing, plus the unused endpoint-row styles) is DELETED. Why: it was a bespoke button system living inside one component; if a smaller button size is ever genuinely needed it must be a Button prop, never a per-component style block. Clicks stop at the container so canvas node presses never bubble into node selection/drag. - StatusHeader: split into composable pieces — StatusHeaderInfo (state dot + name + state + status subtitle) and StatusElapsed ("Started Xs ago") — so Card-based hosts can place them in the Card's header/headerActions slots instead of nesting a second header row. StatusHeader remains as the self-contained composition for non-Card hosts. StatusActions now renders a stock <Button small> (danger for Stop, primary for Run/Subscribe) per the Card-header rule that header buttons are plain Button small. - SourceSection: the section shell is now a real stock Card — StatusHeaderInfo in the header slot, the action cluster + elapsed in headerActions, the pill bar as the Card toolbar, and the pane body + PlayBar as an edge-to-edge body (noBodyPadding). Why: the section was spreading commonStyles.card and hand-building everything the Card component already owns (header row, divider, actions cluster), which is exactly how the Endpoint Info button ended up bespoke in the first place. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(vscode,shell-ui): task arguments + debug output become per-task pipeline settings Move "engine arguments" and "debug output" out of the per-connection-group local-engine settings and re-introduce them as per-task pipeline settings (taskArguments, pipelineDebugOutput) applied through `.use` at task start. Why: the old knobs were ENGINE PROCESS flags — changing tracing or adding an argument meant tearing down and respawning the local engine, and the settings were meaningless against a cloud connection (no local process to flag). But both knobs are really properties of a RUN, not of the engine: the engine already accepts per-task args via `.use`, and --trace=debugOut belongs to the task being debugged. Modeling them per-task makes them work identically on local, on-prem, and cloud connections, and takes effect on the next run instead of the next engine restart. Details: - config.ts: taskArguments + pipelineDebugOutput join the pipeline settings group (with getTaskArgs()); the local connection group keeps only engineVersion. The raw argument string is intentionally passed as a single entry — the engine shell-splits it, so quoted paths survive. - engine-local.ts: the engine now spawns with base args only; no per-task flags leak into the process command line. - debugger/session.ts: launch requests attach getTaskArgs() so F5 runs get the same per-task arguments as panel-started runs. - extension.ts: the engineArgs array->string migration and the old local.debugOutput/engineArgs key migrations are deleted with the keys themselves — there is nothing left to migrate into. - package.json/PipelineSettings/SettingsWebview/panels: the settings UI moves the two fields into the Pipeline section, adds enum descriptions and explicit ordering so the webview and the contributes.configuration model stay mirror images of the cloud app manifest. - shell-ui: SettingSchema documents that `enum` stays string[] per the frozen v0 contract while integer schemas (pipeline TTL) may carry numeric entries in manifest JSON at runtime — renderers go through String() and coerce back via `type`, which is what SettingsPage now does. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(shared-ui): stock Button gains the mini (16px) tier; canvas node actions use it The stock-Card migration replaced PipelineActions' private micro buttons with Button small (26px). That is correct in card headers but overflows the canvas source-node card: "Drop now" + "Endpoint Info" side by side are wider than the node. Per the rule that killed the micro tier in the first place — a smaller button size must be a Button prop, never a per-component style block — the old sizing (16px tall, 9px font, 0 6px padding, radius 3, weight 500) returns as a stock `mini` prop, and PipelineActions renders mini in both of its hosts. `mini` composes after `small` so it wins if both are set. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(run-logging): DVR v2 — client log sessions, permanent trace identity, live/replay parity One coherent layer replaces the ad-hoc "subscribe + backfill" model: every consumer of task events now reads from a LogEventStream SESSION over the run-log continuum, and live delivery is just the last stop of the same walk. == Client session (log-stream.ts / log_stream.py, both SDKs in lockstep) == - Live events are NOT a special case. Arrivals append to a "live bucket" that is an ordinary CachedSegment with a sentinel id sorting after every server segment, so history walks, trace collection, and status folds all traverse disk segments and the live tail through one code path. INVARIANT: the live bucket is never discarded — only prefixes proven to be server-backed leave it. When the bucket exceeds 24MB the catalog is reloaded and the prefix covered by sealed segments is dropped; extending the last disk segment advances maxDiskSeq the same way. Why: any policy that could throw away a live event that never reached disk silently loses data at exactly the moment the user is watching it happen. - Two delivery modes, no pump/poll machinery: HISTORY walks the cached segments in order until exhausted, then flips to LIVE; in LIVE mode an arrival appends to the bucket AND is delivered directly to the session callback (when pinned + playing + past the watermark). The walk cursor dedupes the overlap by seq. Why: the previous arrival-drop/poke iterations had windows where events sat in the bucket with no deliverer — the two-mode model has none, and disk state is irrelevant to live delivery. - Segment cache is capped at 6 resident segments, evicting least-recently USED (touched on ensure and on pump serve) — farthest-from-needle eviction would evict exactly the segments a replay is about to need again. - Sessions self-register their monitor (all event classes) on open and deregister on close, refcounted. Why: consumers were silently missing OUTPUT/TASK-class events because the host's canvas-oriented monitor (summary/flow) was the only registration; the session knows what it needs and must not depend on what some host happened to subscribe. == Trace identity (contract) == - A trace IS its begin event's continuum seq. Slot ids recycle within a run and fold docIds are synthetic per-client counters — neither survives a refresh, both collide across runs. The begin seq is epoch-microsecond seeded and globally unique for the life of the stream, so getTrace(beginSeq) is deterministic from any position, live or replay, in both SDKs: span-table binary search to the owning segment, then collect the slot's flow events forward to its end, crossing segments and the live bucket, with extend-on-miss for a still-active tail segment. - SSE events ride along: the writer records which segments an in-flight document's SSE touched, and trace collection includes apaevt_sse rows by pipe_id, so node narration ("embedding chunk 3/10") appears inside the call tree it belongs to. == Server (task_engine.py / run_log.py) == - _forward_task_event stamps project_id + source into every task-scoped body centrally, replacing per-event-builder stamping and envelope-id suffix tricks. Why: clients must be able to filter membership with a pure body test; identity established at ONE choke point cannot drift per event type. - apaevt_task begin carries beginSeq — the run-begin lifecycle marker's seq (the marker itself is log-only). Clients synthesize the chapter locally from the announcement and join the server's chapter list by exact seq equality. Why: a task-begin-triggered chapter refetch is both a race (the control file may not be rewritten yet) and a thundering herd (every watching client fetching at once); carrying the identity in the announcement makes the local fix-up exact and free. - Teardown order: the run log closes LAST, after the final zeroed status and the terminal task/lifecycle forwards. Why: the log is the record; closing it before the terminal events made every recorded run end without its end. == UI (shared-ui project module) == - useTaskEvents consumes a session instead of listening to host events: fold with runway throttling, FOLD_CAP honesty, and DEAD ZONE = CLEARED — outside any chapter, every pane reports empty/zero at ANY position, live or replay. Why: stale panels in the gap between runs read as data from a run that never happened. - |< seeks to the current chapter's begin (or the previous chapter's end) and >| to the chapter end (or next begin), with an epsilon so landing on a boundary resolves into the intended track. - SourceSection owns the session lifecycle, keeps the local timeline fixed up from task begin/end announcements (server list reconciled by the slow poll only), and hosts the trace DetailPanel: right-docked, width persisted, Prev/Next over the request list, TraceDetail fetching by beginSeq. - Trace pane becomes a pure request list; TraceDetail is a standalone component (also opened from Analyze) rendering the full call tree with Data|JSON|Raw stock ToggleGroup and interleaved SSE narration rows. - AnalyzePane's slowest-documents rows open the trace popup instead of seeking — inspecting WHY a document was slow is the point of the table, and the begin seq each row already carries is the popup's exact key. - LogPane caps its projection honestly (notices for both the display cap and the fold window) and downloads via a full continuum walk, so the file is complete even when the screen is not. - PlayBar colors the live block from task begin/end announcements (runActive) rather than timeline completion guesses, with a square live edge. == Tests == - test_log_stream.py: golden suite proving play==read equivalence, splice and console/status exactness, getTrace by begin seq across recycled slots, SSE inclusion, active-segment growth, direct live delivery, raced-fetch delivery, the 24MB split, and the LRU cap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(run-logging): address PR 1661 review findings — pump pacing, seam drain, epoch races, vscode event routing Fixes for the verified review findings on the run-logging PR. Each carries the why; two findings were rejected (reasons on the PR threads: the conftest stubs are session-scope shims by design, and migrating engine-process args into per-task args would inject semantically wrong flags). - log-stream.ts / log_stream.py: timed playback (speed > 0) stalled forever on the first inter-event gap — the pacing branch slept/rescheduled and then recomputed the SAME delay from an unchanged playback clock. The clock now advances past the wait (epoch-guarded in TS so a retired timer cannot touch the live clock). The review caught the Python copy; the TS setTimeout re-entry had the identical stall and is fixed to match. - log-stream.ts / log_stream.py: an event arriving during the history-walk's final awaits (nextEvent exhaustion -> refreshTimeline) was buffered into the live bucket but never delivered — ingestLive only delivers arrivals AFTER the pin flips, and the walk was already done. The pump now synchronously drains bucket events above the watermark right after pinning; no await can interleave an arrival mid-drain in either runtime. - log-stream.ts: seek() now retires any in-flight pump (pumpEpoch bump) before its repositioning awaits — stopTimer only kills a SCHEDULED re-entry, so a pump parked on a fetch could resume against the old epoch and mutate watermark/basePos or fire a stale callback interleaved with the new position. Pump post-await guards also re-check playing/closed, so a pause landing mid-fetch cannot leak one extra delivery. (Python is immune: its seek cancels the pump task outright.) - vscode ProjectProvider/ProjectWebview: the host monitor subscribed only summary/flow and forwarded only status/flow events, so the webview's live log and run-active timeline never received output/lifecycle events. The monitor now adds output+task, handleEvent forwards every stamped event for the editor's project, and the webview's membership check becomes the pure body.project_id test (the envelope source-suffix hack is deleted — the server's central identity stamp made it obsolete). - vscode SidebarProvider: sidebar-started runs now pass the same pipelineTraceLevel + ttl as editor launches — a pipeline must run identically regardless of where it is started. - shared-ui utils.ts: parseServerEvent skips flow events missing the continuum header stamps instead of emitting undefined eventTime/seq into the folds (NaN timestamps, unstable ordering). A skip, not a wall-clock fallback: the folds are event-time-sourced by design; against a pre-continuum server the trace pane is empty rather than corrupt. - useTraceState.ts: the fold-reset effect reads resetKey but did not list it — the reset only fired because both callers change the traceEvents reference alongside it. The dep is added; the ref compare keeps re-runs idempotent. - docs guide/methods/log.md: both getTrace examples passed the recycled pipe slot (.id) instead of the trace identity (beginSeq), contradicting the contract stated in the same file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(run-logging): the DAP envelope is protocol again — continuum stamps live in the body, and only there The continuum seq was riding in the DAP header `seq` slot. That violates DAP: `seq` is one monotonic counter per connection, shared by requests, responses, and events on that socket. With per-task continuum counters, a client monitoring two tasks interleaved two unrelated sequences (event Y followed by event Y-555), and even one task mixed the connection's small counter with continuum values. Correlation (request_seq) was never broken, but event-stream sequencing was broken protocol. The fix draws a hard line: the DAP envelope is TRANSPORT, the body is the RECORD — and the record has exactly ONE representation. - dap_conn.py is byte-identical to develop — send_event(event, id, body), nothing else. A generic DAP handler carries no logging concepts; the branch's earlier stamp-forwarding parameters are gone. Every connection mints its own envelope seq; the monitor forward is plain send_event, and the shared body carries the stamps to every subscriber identically. - stamp_log_event writes body.eventTime + body.logSeq, beside the project_id/source identity already stamped there. Nothing nonstandard ever touches the envelope: eventTime as a header field was itself a spec extension, so both stamps moved into the body together. - NO mirroring anywhere: clients read body.eventTime/body.logSeq directly. The SDK LogEvent (TS + Python) and shared-ui's TaskEventMessage lost their top-level stamp fields — the body is typed (LogEventBody / TaskEventBody) with the stamps required, and every internal walk (watermarks, pacing, binary searches, folds, getTrace, seam drain, maxDiskSeq) reads the body. Why no mirror: a second copy of the stamps is a second thing that can be wrong, and the mirrored type read like a wire claim — the type said "header eventTime" while the wire said otherwise. One representation cannot drift. - Legacy compatibility is normalization INTO the body: codec decode canonicalizes old header-stamped v2 segments (body.eventTime/logSeq filled from the header once), so old recordings read identically to new and no consumer carries a fallback. - Host gates (rocket-ui ProjectProvider, vscode ProjectWebview) admit an event on one pure body test — identity + stamps present — and pass the message through untouched. - Seeding is catalog-based: a fresh stream starts at 1; open() lifts the floor to control.lastSeq + 1, so the continuum continues exactly where the recorded stream left off. The epoch-microsecond seed, the max(clock, lastSeq) logic, the clock-anomaly marker, and CONST_LOG_SEQ_FLOOR are deleted. Why: the writer reads the control file at open anyway, so the monotonicity guarantee is free, and small human-readable seqs beat 16-digit epoch values. The crash window (an unpersisted tail re-issuing values) is accepted: a crash drops every websocket, clients reconnect with fresh sessions and fresh live buckets, so nothing stale survives to collide. - shell-api cleared and REFROZEN at v0 via `./builder shell:freeze` (sanctioned: nothing has been released, so no consumer ever saw the old contract). The frozen LogEvent is the body-only shape; SHELL_API_VERSION restamped to 0; all accumulation files regenerated by the freeze script. - Tests: the fake stamp mirrors body semantics; golden assertions read through body accessors (seq_of/time_of); the clock-anomaly test dies with the behavior it tested; fresh-stream seqs assert from 1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(run-logging): live-minimal PlayBar, RocketRide banner in every run log, terminal-tight log lines Three polish items from live testing: - PlayBar: pinned-live-and-hands-off is now just the bar — the pause button, the LIVE clock readout, and the zoom scale fold into the hover (which also unfolds the full lane, as engagement always has). Replay counts as engaged, so its chrome never hides. The collapsed live bar is a little taller (16px vs the 10px replay ribbon): with the chrome gone, the bar is the only presence at the live edge, so it gets more of one. - The RocketRide ASCII banner now opens every run's log. It lived only in ai/web's WebServer.serve() — the EaaS operator console — and the task node never runs a web server (node.py hands the process to rocketlib), so the run log never saw it. The art moves to a shared ai/logo.py; WebServer imports it and node.run() prints it as its first act on stdout, which IS the console the run log records. - Log pane line height 1.55 -> 1.25: log output (and the banner art) reads as a terminal, not prose. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix(run-logging): address PR 1661 review round 2 — finite stamps, span keying, legacy read() canonicalization, DBG label safety All fourteen review findings verified and fixed (none rejected this round): - Stamp guards are Number.isFinite / math.isfinite everywhere a wire event is admitted: parseServerEvent, both host gates (rocket-ui ProjectProvider, vscode ProjectWebview), and BOTH SDK ingest paths — typeof admits NaN/Infinity, and Python's ingest additionally required only logSeq, so a time-less event would have folded at timestamp 0 into every seeded read. - AnalyzePane keys open enter/leave stacks by slot:component. Concurrent documents run the same components on different slots; a component-only stack paired one document's enter with another's leave and corrupted the latency totals. Reentrancy within a slot still stacks, like the trace fold. - The server reader canonicalizes legacy header stamps INTO the body before returning events, so everything read() yields satisfies the body-stamp contract — consumers never carry a header fallback. This matches what the session codec already did on the segment path. - DBG tool-trace labels can no longer break the wire format: '*' is the DBG field delimiter and only the final JSON field tolerates it (maxsplit), so the no-node call label is now a bare 'tool()' (no placeholder — reads naturally in the trace tree) and _emit_tool_trace strips '*' from the component field defensively. The wire format itself is unchanged (shared with the C++ debugger). - vscode migrations fold the four legacy *.debugOutput keys into the one per-task pipelineDebugOutput (OR per scope — any scope that had it on keeps it on) and clear the legacy keys. engineArgs stays unmigrated: engine PROCESS flags are not per-task arguments, and injecting them into every .use call could break runs. - LogPane downloads attach the anchor to the document for the click and defer the blob-URL revoke — a synchronous revoke can cancel a download that has not begun fetching (same pattern as the data-grid export helper). - summaryToolInput only renders no-args for absent input or an empty plain object — falsy scalars (0, false, '') are real arguments and show as such. - SettingSchema.enum widened to Array<string | number> (workspace types + app-sdk) — integer schemas carry numeric choices; rendering keeps the String() coercion. - Docs: client.log gains its co-located Python guide (client-python/docs/log.md); the getTrace failure contract in both SDK sessions now names both failure modes (below horizon, or no trace-begin at that seq); TraceEvent.seq loses the stale epoch-us wording. - shell-api v0 cleared + refrozen via `./builder shell:freeze` (pre-release procedure) — the frozen surface picks up the widened enum, the canonicalized read() contract, and the corrected getTrace docs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(vscode): debug-output migration gets its own marker and loss-proof ordering Review findings on the migration added in the previous round: - It lived inside the settingsMigrationV2Done block, so any user who had already completed the V2 migration would skip it forever. It now runs under its own debugOutputMigrationDone marker, independent of V2. - It deleted the legacy keys while collecting them, before writing the replacement — a failure mid-migration would have destroyed the only copy of the old configuration. The order is now collect, write pipelineDebugOutput, then delete the legacy keys: retry-safe at every step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(vscode): remove the Rocket Ride: Console output channel The channel mirrored every server output event into VS Code's Output panel. The task's Log pane now renders the same events (run-log backed, live and replay), so the mirror was a redundant second copy with none of the log's structure. Removal, end to end: - connection.ts no longer writes output events to the channel; the shell:event emit that feeds the webview panes is untouched, and non-output apaevt_* diagnostics still log to the main Output channel. - The channel and logger.console() are deleted; engine spawn/readiness lines (engine-process diagnostics, not task output) move to the main Output channel. - The sidebar's wildcard monitor narrows from ['task','output'] to ['task']: its output subscription existed solely to feed the mirror, so the server no longer broadcasts every console line to the extension for a consumer that no longer exists. Per-editor monitors subscribe to output themselves for the Log pane. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style(docs): ruff format markdown code blocks to unblock CI The Ruff gate runs `ruff format --check` with docstring-code-format enabled, which formats the python code fences inside our docs. 14 doc files carried unformatted snippets (mostly pre-existing, not introduced by this branch) and were tripping the check on this PR. No source code changed — docs only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015nTVr6jfSFYm1GppxbjghP * fix(task,docs): accept the bare >JOB control (empty status); executable doc examples - transport_stdio: a bare '>JOB' (no delimiter) is the empty job status — the engine's monitorStatus('') only appends '*text' when there is text, and nodes use the empty form to CLEAR the status line (e.g. the sentence transformer after model load). It now forwards as an empty apaevt_status_message instead of falling through to the unknown-control branch and dumping 'Unknown control: >JOB' into every run's console. - Python SDK docs (review round 4): the index.md context-manager example is now executable (async main + imports + asyncio.run — async with cannot appear at module scope), and the log.md closed-trace example guards traces['closed'][0], which is legitimately empty on a fresh or all-in-flight stream. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Dmitrii Kataraev <dmitrii.kataraev@rocketride.ai>
* feat(nodes): add tool_pipedrive CRM tool node
Exposes the Pipedrive REST API v1 to agents: 256 tools across 24 resource
groups (deals, persons, organizations, activities, pipelines, stages, notes,
leads, products, fields, files, users, roles, permission sets, teams, goals,
filters, webhooks, subscriptions, mailbox, call logs, projects, misc), plus a
generic `request` tool that reaches any remaining v1 endpoint through the same
auth, rate-limit and read-only layer.
Full coverage is more tools than an LLM can choose between reliably, so
`pipedrive.toolGroups` controls which groups are published. The default eight
core CRM groups publish 108 tools; `["all"]` publishes everything. Filtering
happens in `IInstance._collect_tool_methods()`, so a gated tool is invisible to
`tool.query` and refused by `tool.invoke` alike.
Client details specific to Pipedrive:
- `{"success", "data", "additional_data"}` envelope is unwrapped; `success:
false` on an HTTP 200 is raised as an error.
- `x-ratelimit-reset` is the seconds remaining in the window, not an epoch
timestamp as in the GitHub client this otherwise mirrors.
- Offset pagination (`start`/`limit`, max 500) everywhere except projects,
which use cursor pagination.
- Personal API tokens go in the `api_token` query parameter; JWT, long or
`Bearer `-prefixed values go in an Authorization header.
- Custom fields are grouped under `custom_fields` on read and written through
the `extra` passthrough that every create and update tool accepts.
Tests: 81 unit tests that need no credentials or network, plus an env-gated
live suite behind PIPEDRIVE_API_TOKEN, with record-creating tests additionally
behind PIPEDRIVE_ALLOW_WRITES.
Also corrects the stale node/service counts in docs/README-nodes.md
(88 -> 118 directories, 118 -> 148 services).
Closes rocketride-org#1675
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes): use the real Pipedrive brand mark for the tool_pipedrive icon
The first icon was a green uppercase "P" next to an unrelated dark dot. The
actual Pipedrive mark is a solid green disc carrying a white lowercase "p"
whose counter shows the disc through it.
Redrawn as plain geometry (disc r=128, stem 36x150, bowl r=56, counter r=27)
so it stays crisp at 16px with no gradients, clip paths or embedded raster.
Uses the default nonzero fill rule with the counter arc wound opposite to the
bowl. Under evenodd the stem and bowl would XOR where they overlap and punch a
green notch out of their junction; nonzero merges them solid while the opposing
winding still renders the counter as a hole.
Keeps the viewBox (SVGR runs with dimensions:false, so an icon without one
computes to zero width) and keeps two distinct colours, which makes the
auto-currentcolor svgo plugin treat it as a multicolour brand mark and pass the
authored colours through untinted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes): trace the tool_pipedrive icon from the supplied brand artwork
The previous version approximated the mark with circles. The real letterform
differs in three ways that are visible even at 16px: the stem carries a spur at
the top left, the bowl is an ellipse rather than a circle, and the counter is an
ellipse tangent to the stem's right edge.
Geometry was measured off the supplied 272x272 artwork rather than eyeballed —
connected-component analysis to isolate the counter, then a least-squares fit of
the bowl ellipse to the glyph's right-hand edge (max residual 2px). Rasterising
the traced primitives and diffing against the source leaves 1.47% of opaque
pixels differing, all of it a sub-pixel outline along the shape boundary.
Still a two-colour multicolour mark, so the auto-currentcolor svgo plugin passes
the authored colours through, and still nonzero winding with the counter wound
opposite the bowl so the stem and bowl merge solid while the counter stays open.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(nodes): warn when tool_pipedrive publishes too many tools for an agent
Full coverage is 256 tools. An operator can already reach that by selecting
groups, and nothing told them the agent's tool choice degrades long before it.
Adds a guard rail that counts published tools rather than groups, because group
sizes are lopsided: deals is 28 tools and permission_sets is 3, so a group count
bounds nothing useful. The threshold is 120, just under the 128-function cap
some providers impose on a single request and above the 107 the default groups
publish.
Crossing it warns and carries on. The tools are still published, in both
validateConfig (so it shows in the config panel while editing) and beginGlobal
(so it shows at pipeline start). Silently dropping tools an operator asked for
surfaces later as "the agent won't do X", which is worse than a noisy panel. The
warning is deliberately not raised from _collect_tool_methods, which runs on
every tool.query and tool.invoke.
"all" is exempt: it is already an explicit opt-in to the whole surface, and
scripted callers that never route through an LLM legitimately want it. Listing
every group by name is not exempt.
Group sizes are derived from the __pipedrive_group__ stamps rather than a
hand-maintained table, so a new tool cannot drift out of sync; a test asserts
every group in ALL_GROUPS has a count. Another test pins the advisory semantics
so the warning is not later "fixed" into a silent truncation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ai): flatten provider content blocks to visible text
Anthropic with extended thinking returns `content` as a list of typed
blocks, not a string. Every non-streaming path handed that list straight
to callers, who stringified it — so the answer arrived as the blocks'
repr, carrying the raw reasoning and its base64 signatures.
For agents it was worse than cosmetic: the ReAct parser reads output back
with a greedy `Action Input: (.*)`, so the serialized wrapper's trailing
`", "type": "text"}]` landed inside the tool arguments and every declared
field then reported as missing.
Extract the block vocabulary that only the streaming path knew into
`ai.common.utils.flatten_content_blocks` and route chat's invoke, the
stream fallback, and the agent's `extract_text` through it. The streaming
path now shares that vocabulary instead of duplicating it, so the paths
cannot drift; the fallback also forwards recovered reasoning to
`on_reasoning_chunk` rather than dropping it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes): make CrewAI tool calls and hierarchical kickoffs survive
Three defects made the CrewAI nodes fail on runs that should have worked.
Tool arguments were dropped. `_ToolInput` declared a single `input` field,
and CrewAI's argument filter intersects the model's emitted keys with the
schema's property names — so every real parameter was discarded before
validation, which then reported all required fields missing. Generate a
typed model per tool instead: JSON Schema types map to real annotations
(so pydantic coerces "123" -> 123), optional fields accept an explicit
null, and paramless tools get an empty schema. The JSON schema is no
longer appended to the description either; CrewAI renders args_schema into
the prompt itself, and the two copies disagreed.
`HostInvokeLLM` never declared `supports_function_calling()`. CrewAI
defines it on `LLM` but not on `BaseLLM`, and several call sites invoke it
unguarded — converter.py, tool_usage.py, reasoning_handler, and
task_evaluator. The AttributeError was swallowed by converter's broad
except and re-reported as "Failed to convert text into a Pydantic model".
Declare it False: the host channel is text-in / text-out and cannot honour
native tool calls, so every call belongs on the ReAct path. Pin crewai <2
for the same reason — 1.15.x moves the BaseLLM contract.
Hierarchical crews then died in planning. `Crew(planning=True)` runs a
task with `output_pydantic` set and drives it with the host wrapper, which
cannot promise JSON; unparseable plan text dead-ends the converter and
raises "Failed to get the Planning output". Make planning opt-in via a new
**Crew Planning** field, off by default.
Also clear `Task.tools` after construction: CrewAI back-fills them from
the task's agent, and a hierarchical crew resolves the executing agent's
toolset for the manager — which let the manager work a delegate's tools
itself instead of delegating.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes): move tool_pipedrive search to the v2 API
Pipedrive retired the v1 search routes: `/persons/search` and its siblings
answer 404 "Unknown method ." and `/itemSearch` answers a plain 404, so
every search tool the node published was dead. Non-search v1 endpoints
still route normally, so only the search calls move — a blanket swap would
break the many endpoints with no v2 equivalent.
Adds a v2 base URL (company-domain aware, sharing v1's normalisation) and
a v2 envelope wrapper: v2 replaced offset pagination with an opaque
`next_cursor`, which `paginated` would have missed entirely and reported
as a single page. Search tool schemas now advertise `cursor` instead of
`start`, and `item_search_by_field` follows v2's renames — `field_type` ->
`entity_type` with the "Field" suffix dropped from each value, and
`exact_match` replaced by the three-way `match` mode.
Also turns Tool groups into a multi-select dropdown with per-group tool
counts, so the field no longer requires typing group names from memory.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(shared-ui): keep live panes working without a host event stream
PR rocketride-org#1661 rewired every accumulating pane — Trace, Errors, Status, Tokens,
Flow, Log, Analyze — to read through a DVR session obtained from the
host's `openEventStream`, a prop no host in this repo binds. Hosts that
have not bridged it across their transport (the VS Code webview is one)
passed a null session, `useTaskEvents.ingestLive` dropped every event, and
the panes rendered their empty states mid-run with no error anywhere.
Fall back to an in-memory store over the live events the host already
delivers. It satisfies the same session contract `deliver` expects — a
pure append, each event handed over exactly once, oldest first, across
repeated `play` calls — and is keyed per stream identity so the several
views `SourceSection` opens all share one buffer. Replay of recorded runs
still requires the real stream.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(test): stop node packages being shadowed by same-named test dirs
nodes/test is a package whose subdirectories are named after node packages
(text_output/, response/, image_cleanup/, telegram/, tool_git/, ...). Two places
put that directory on sys.path so they could bare-import a test-root module:
conftest (for _sys_modules_guard) and cloud_tts/test_cloud_tts (for
test_contracts). Once nodes/test sits ahead of src/nodes, `import text_output`
resolves to the *test* package, whose __path__ holds no node module — so
`from text_output.instance import Instance` dies with ModuleNotFoundError.
Whether it bites was pure collection order. The affected modules bootstrap with
"insert src/nodes only if absent", which never re-prioritises: previously nothing
seeded src/nodes before cloud_tts (c) inserted nodes/test, so image_cleanup (i)
was the first to insert src/nodes and landed it in front. The new agent_crewai
tests seed src/nodes during collection of agent_crewai (a) instead, so cloud_tts
now stacks nodes/test on top of it and the guard short-circuits in all four later
modules — four collection errors, none of them in code this branch touches.
Import both test-root modules package-relative and drop the sys.path inserts, so
nodes/test never shadows a node package. cloud_tts gains the __init__.py the rest
of the node-named test dirs already have, which is what makes the relative import
resolvable. The four bootstraps now move src/nodes to the front unconditionally
rather than skipping when it is already somewhere on the path, and the guard suite
pins both invariants: nodes/test stays off sys.path, and a node package imported
by name resolves under src/nodes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes, shared-ui): address CodeRabbit review on the tool_pipedrive PR
Security
- pipedrive_client._company_host accepted anything up to the first slash, so a
configured domain of "evil.example#" built https://evil.example#.pipedrive.com
and Requests sent it — with the api_token or Authorization header attached — to
evil.example. Cut at the first path/query/fragment delimiter, refuse userinfo
and ports, lower-case, and require a single DNS label; anything else falls back
to the generic host.
- Free-form ids (lead uuids, permission-set ids, channel ids, goal ids, note
comment ids) were f-stringed into request paths unencoded, so a value carrying
"/", "..", "?" or "#" retargeted the call at another endpoint — for the DELETE
tools, a destructive one. Added _base.path_segment() and applied it at all 20
string-id sites.
- Pin requests>=2.34.2 to match constraints.lock, as tool_cognee/tool_github/
tool_n8n already do.
Correctness
- crewai_base._annotation_for passed prop['type'] straight to dict.get. JSON
Schema spells a nullable field as ["string", "null"], and an unhashable key
raises TypeError instead of returning the default — one such property aborted
the whole _build_crew_tools loop. Non-str types now fall back to Any, which is
what the docstring already promised.
- deal_followers_users_list GET the same /deals/{id}/followers as
deal_followers_list and only differed by running follower rows through
clean_user, emitting a user-shaped object built from mismatched fields.
Dropped it; resolving followers to users needs a /users/{id} call per follower.
- add_time (deals, organizations, persons) and origin_id (leads) were in the
write-key tuples but not in the matching schema props, so body_from could never
populate them. Added the properties.
- product_variation_update and both project-plan updates passed the builtin dict
as the response cleaner: dict(None) raises TypeError on an empty 204 body.
Switched to passthrough, as every other write already uses.
- liveEventSession.getStatus scanned from the live edge, contradicting the
documented "as of the session position" and the caller, which seeks to an
anchor first — a repositioned view seeded a replayed run with the current
status. Scan back from the watermark.
Maintainability
- Eight bulk deletes each repeated the ids guard, the CSV build and a trailing
_require_write(), and left ids optional so a missing argument surfaced as a
hand-rolled ValueError. Added _base._delete_bulk (write gate first), marked ids
required in every schema, and delegated all eight.
- projects._cursor_list re-implemented paging_params_v2 and the v2 envelope
shaping; it now uses the shared helpers, so the cursor contract has one
implementation instead of two that can drift.
- file_list advertised the shared 1-500 range, but /files documents a maximum
limit of 100. PAGING/paging_params/_list take a max_limit override and file_list
passes 100.
- _build_crew_tools' docstring still claimed the JSON Schema is embedded in the
tool description, which stopped being true when args_schema took over.
- Tests: 'ACME'.lower() evaluated to 'acme' before the call, so the case path was
never exercised; test_tools imported pipedrive_client bare, giving the module a
second identity in a run that also collects test_pipedrive; a vacuous
b64encode assert, a missing empty-pipelines guard, and a hard-coded field_id
of 1 that fails rather than skips on a fresh sandbox.
Two findings were not applied. `extra` is not dropped by fields/misc — body_from
merges args['extra'] by default (tools/_base.py). And the @patch/parametrize
signatures are already correct: mock injects positionally innermost-first while
pytest passes parametrize values by keyword, so the suggested reorder would bind
the mocks to the parametrized names.
New tests cover the rejected domains, path-segment encoding, non-str schema
types, the write-gate ordering, and the watermark-bounded status.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes): harden the tool_pipedrive path and bulk-delete helpers
Two follow-ups from the second CodeRabbit pass, both on helpers added in the
previous commit.
_delete_bulk merged the caller's `extra` pass-through after assigning the
validated `ids`, so `deal_delete_bulk({'ids': [1], 'extra': {'ids': '2,3'}})`
deleted 2 and 3 instead of 1. Merge `extra` first and let the validated CSV win.
The hand-rolled version this replaced had the same ordering, so the flaw predates
the helper.
path_segment relied on quote() alone, but "." and ".." contain nothing quote
escapes — `/notes/{id}/comments/..` survived encoding intact and addressed the
parent resource. Reject those two segments instead of encoding them.
Tests cover both: a dot-segment id raises before any request is made, and a
pass-through "ids" no longer displaces the validated list while other extra
parameters still reach the query string.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(nodes, ai): keep the Pipedrive token out of transport error text
The personal-token path puts the credential in the `api_token` query parameter,
so requests bakes it into the URL. A transport-level failure (DNS, refused
connection, TLS, timeout) raises a RequestException whose text carries that URL,
and `raise ValueError(f'Pipedrive request failed: {exc}') from exc` copied the
token verbatim into the error the agent framework surfaces and logs. The chained
cause held the same URL, so scrubbing the message alone would still have leaked
it through any printed traceback.
Build the message from the exception type plus method and path — all secret-free,
and the type still distinguishes ConnectTimeout from SSLError from ConnectionError
— and raise `from None`. Verified this was the only such site: the JSON-decode
path stringifies `resp.text[:200]` rather than the URL, `_raise_pipedrive_error`
builds from parsed response fields, and the tenacity wrapper uses `reraise=True`
so no RetryError repr is introduced.
Three tests pin it, and all three fail against the previous code: the token
appears in neither the message nor the formatted traceback (that half is what
pins `from None`), and the message still names the failure and the call.
Also:
- notes' `add_time` was in _NOTE_WRITE_KEYS but not _NOTE_WRITE_PROPS, so
body_from could never populate it — the same class fixed for deals,
organizations, persons and leads in 7b3cc53. A scan of every
*_WRITE_KEYS/*_WRITE_PROPS pair confirms notes was the last one.
- ChatBase._chat had no direct test. flatten_content_blocks and extract_text are
covered, but nothing pinned the wiring that routes the provider's content
through them, which is the contract every non-streaming chat caller depends on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…mented ingester protocol (zero engine changes) (rocketride-org#1612) * feat(sdk): rocketride.otelbridge — OpenTelemetry export over the documented ingester protocol (rocketride-org#1609) Maps the engine's monitor event stream to standard OTel telemetry with zero engine/server changes — the client-side refinement invited by tracking issue rocketride-org#616 after the engine-invasive rocketride-org#498 was archived. - mapper.py: stateful FlowSpanMapper reconstructs span trees from apaevt_flow — task root span, pipe span per pipe id, component child spans paired by component identity (per the observability doc's warning against stack-position pairing); trace.error -> span status ERROR + exception event; apaevt_sse -> span events; unclosed spans at end/shutdown flagged rocketride.span.unclosed=true; per-pipe state is bounded. MetricsMapper maps apaevt_status_update snapshots to OTel instruments (object counts, rates, cpu/gpu memory + peaks). - Semantic conventions per the July 2026 semantic-conventions-genai snapshot: gen_ai.provider.name (not the deprecated gen_ai.system), gen_ai.operation.name from the well-known list only, span names '{operation} {model}' style, INTERNAL span kind for in-process component executions. - Privacy by default: lane payloads / prompt / result content never reach attributes or events unless include_content is set (8KB truncation cap when it is); enforced by a sentinel-scan test across all exported attributes and events. - setup.py: TracerProvider/MeterProvider + OTLP http/protobuf exporters; ALL opentelemetry imports are lazy so the base package imports without the extra. 53 unit tests (InMemorySpanExporter/InMemoryMetricReader; importorskip so the base CI matrix stays green), including a full replay of the captured real-engine event fixture. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(cli): add 'rocketride otel' subcommand + optional [otel] extra (rocketride-org#1609) rocketride otel [--endpoint URL] [--protocol http|grpc] [--service-name] [--headers k=v,...] [--include-content] [--no-metrics] + the standard connection args. Subscribes via rrext_monitor with token '*' (the observability doc's recommended ingestion scope, types TASK|SUMMARY| FLOW|SSE), dispatches to the mappers, reconnects with capped backoff and resubscribes (subscriptions are per-connection and not durable), and flushes exporters on SIGINT/SIGTERM. Endpoint resolution honors the standard OTLP env contract: explicit --endpoint > OTEL_EXPORTER_OTLP_TRACES_ENDPOINT (verbatim) > OTEL_EXPORTER_OTLP_ENDPOINT (base URL + /v1/traces|/v1/metrics) > default. Packaging: opentelemetry-sdk + OTLP http exporter live behind an optional extra — the base install gains zero dependencies, and running the subcommand without the extra exits 2 with the exact "pip install 'rocketride[otel]'" hint before any connection attempt. 25 tests: config precedence matrix, dispatch/reconnect/resubscribe with a fake client, shutdown flush ordering, missing-extra exit path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: OpenTelemetry bridge guide with Jaeger, Langfuse, and Datadog recipes Setup guide covering the span model (task -> pipe -> component tree), the privacy default and --include-content, the FLOW-requires-trace-level callout, reconnect semantics, and copy-paste recipes with exact OTLP endpoints/auth for Jaeger all-in-one, Langfuse (/api/public/otel, Basic auth), and Datadog. Notes that Jaeger does not ingest OTLP metrics — pair it with --no-metrics or point metrics at a collector. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): handle partial OpenTelemetry installs — guard the HTTP exporter import CI exposed a real environment our matrix missed: opentelemetry-api/sdk present as another package's transitive dependency (tool_daytona pulls them in) while the OTLP HTTP exporter is absent. Two fixes: - setup.py: the http trace/metric exporter imports are now guarded like the grpc ones — a partial install raises OtelNotInstalledError with the pip install 'rocketride[otel]' hint (CLI exit 2) instead of a raw ModuleNotFoundError. Verified live in a reproduced partial venv. - test_otel_setup.py: importorskip targets the most specific module the tests exercise (opentelemetry.exporter.otlp.proto.http), so the module skips cleanly in CI's partial environment; plus a regression test that simulates the partial install via a None sys.modules entry and asserts the friendly error. Partial venv: 63 passed + 1 module skip; full extra: 79 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): OTel bridge review round — privacy-gate errors, run-id promotion, header precedence, shutdown ordering CodeRabbit review round on rocketride-org#1612, each finding verified before acting: - mapper: trace.error no longer bypasses the privacy gate — by default only the error signal exports (status ERROR, bare exception event, generic description); --include-content restores verbatim text, now subject to the same 8KB truncation strings previously skipped. The sentinel privacy test plants the sentinel in trace.error. - mapper: a run first seen via flow events (before the task 'running' snapshot) is now promoted from its project|source fallback key to the canonical run id when it arrives — previously a second run state was created (broken parentage) and the orphaned fallback state leaked permanently. Confirmed with a failing repro; guard test ensures two distinct canonical runs sharing (project, source) never merge. - config/setup: only --headers become explicit exporter headers; env headers are left to the SDK so OTEL_EXPORTER_OTLP_TRACES_HEADERS / _METRICS_HEADERS signal precedence works (it was silently defeated). Precedence matrix tests added. - setup: meter provider shutdown wrapped in finally so metrics flush even when tracer shutdown raises. - CLI: --trace-level none message now says flow tracing stays disabled; docs recipes pass collector auth via OTEL_EXPORTER_OTLP_HEADERS env instead of argv (shell-history hygiene); error-mapping doc wording matches the implementation (component leave events). Full extra: 90 passed. Partial install (api/sdk without exporter): 69 passed + clean skip. ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): absorb stranded fallback run state when canonical state precedes it Follow-on to the run-promotion fix: an SSE event creates the canonical run state WITHOUT registering the (project, source) alias (SSE bodies carry no project/source per the protocol), so an id-less flow for the same run could still open a parallel fallback-keyed state — and because the canonical state already existed, the promotion branch was skipped, stranding the fallback state (kept alive forever by snapshot reconciliation) with duplicate spans. _get_run now reconciles whenever an event carries both the canonical id and project/source: no canonical state → promote (unchanged); canonical state exists → _absorb_fallback_run migrates non-colliding pipes (open spans re-stamped with the canonical run id), closes colliding pipes as rocketride.span.unclosed=true, and drops the fallback key. Only fallback-format keys are touched, so distinct canonical runs never merge (guard test unchanged and green). Both new regression tests failed before the fix. Full extra: 92 passed; partial install: 71 passed + expected skip; ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): stamp OTel spans with the engine's body.eventTime The bridge was written against a monitor protocol that carried no event timestamps, so every span used the bridge's own clock at event arrival and the guide documented that as a limitation. Since rocketride-org#1661 the engine stamps `body.eventTime` (epoch seconds, set once at ingress by the run-log continuum) on every forwarded event, and derived flow events inherit their source event's stamp — so the limitation is no longer true and the documentation shipped with it was wrong against develop. Span start/end and span-event timestamps now come from `body.eventTime`, making durations engine-measured instead of bridge-observed: WebSocket transport latency and event-loop scheduling no longer inflate every component span. The fallback is exactly the previous behaviour. A body with no usable `eventTime` (an older engine, or a non-float/NaN/non-positive value) and every span closed outside an event — shutdown, seeded-snapshot reconciliation, tracked-run cap eviction — use `time.time_ns()`, the same clock the SDK would have used. Because a run can therefore mix both sources, each open span records its own start and a close is clamped to never precede it, so no span can report a negative duration. Metric points are unchanged: OpenTelemetry metric instruments take no explicit timestamp, so they remain arrival-timed. That is now stated rather than left implicit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(client-python): publish the OTel bridge guide and mirror its CLI entry Two co-location gaps (AGENTS.md:5-28) and one stale claim. The 369-line guide lived at docs/README-otel-bridge.md, and root docs/ is not in the docs-site glob (packages/docs/scripts/lib/gather.js:25), so the guide was GitHub-only. Moved to packages/client-python/docs/otel-bridge.md, the tree already mounted at develop/python (packages/client-python/scripts/ tasks.js:236) alongside analytics.md and log.md, so it is gathered to the site at /develop/python/otel-bridge. Its in-repo links become site routes to match the sibling pages; the three references to it were repointed. packages/client-python/docs/index.md is the site's Python page and mirrors the PyPI README's ## CLI block line for line, but had no `rocketride otel` row and none of the OpenTelemetry paragraph the README gained. Added both. The guide's ## Limitations still said span timestamps are arrival time because the wire carries none; the previous commit made spans engine-timed, so that bullet now describes what actually remains bridge-timed (spans closed outside an event, pre-continuum engines, and all metric points). The Python blocks in the guide are also reformatted: ruff formats Markdown code blocks as of 0.16, and CI's ruff-action is unpinned, so the file failed `ruff format --check` today at either path. Repo-wide check is clean again. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(changelog): record the rocketride otel bridge under [Unreleased] CHANGELOG.md [Unreleased] ### Added is the convention for user-visible additions and is kept alphabetical by scope; a new CLI command and a new optional extra qualify. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(client-python): name Grafana Tempo in the OTel backend recipes Issue rocketride-org#1609's acceptance list says "Jaeger/Tempo", and the guide covered Tempo only implicitly as "Grafana stacks". Tempo is the OTLP receiver a Grafana user actually points the bridge at, and it needs one non-obvious step (enable distributor.receivers.otlp) plus a separate metrics backend, so it is worth naming rather than leaving a reader to infer. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sdk): address CodeRabbit review round on the rebased head Nine threads on the rebased head, all in the OTel bridge. Dispositions: - docs/otel-bridge.md:151 (Sensitive Data Exposure, CWE-522) — FIXED. The Langfuse and LangSmith recipes now prompt for the keys (printf + read -rs, spelled identically in bash and zsh) instead of embedding placeholders that operators replace with live secrets in a command line, and point at a secret manager as the better source. The base64 pipeline is now `printf '%s' ... | base64 | tr -d '\r\n'`: real Langfuse keys make the joined string 85 characters, past GNU base64's 76-character wrap, and command substitution strips only trailing newlines, so the embedded break would have produced an invalid Authorization header. - cli/commands/otel.py:207 (Sensitive Data Exposure, CWE-532) — FIXED. The startup line ran `config.endpoint` straight to stdout; an endpoint can carry userinfo credentials or a signed query string. It is now always passed through `redact_endpoint()`, which drops userinfo, query and fragment while keeping scheme/host/port and the ingest path operators actually verify. - cli/main.py:466, otelbridge/config.py:177, otelbridge/setup.py:123 (Cleartext Transmission, CWE-319) — FIXED, once, in one place. `validate_transport_security()` runs before any exporter exists (in `build_providers`, and again in the CLI so the failure is exit code 2 rather than a traceback): a credential-looking header name from `--headers` or any of the three OTEL_EXPORTER_OTLP_*HEADERS variables, plus a non-loopback cleartext endpoint for either signal, aborts startup. Header VALUES are never read or echoed — only names reach the message. Loopback stays exempt for local collectors; `--insecure` / ROCKETRIDE_OTEL_ALLOW_INSECURE=1 is the explicit opt-out and warns on stderr. - otelbridge/config.py:189 (CWE-201, redirect header forwarding) — FIXED. The OTLP/HTTP exporters now get a session that never follows redirects. `requests` strips Authorization only when the redirect changes host, so a 3xx replayed x-api-key to the target and even kept Authorization on an https -> http downgrade to the same host. - otelbridge/config.py:177 (endpoint precedence) — FIXED. `from_args_env` no longer pre-reads OTEL_EXPORTER_OTLP_ENDPOINT into an explicit exporter endpoint: doing so overrode the signal-specific _TRACES_ENDPOINT / _METRICS_ENDPOINT, inverting the OTLP spec order — the same defect already fixed for the header variables in c23b312. The SDK now resolves all three, and `effective_endpoint()` mirrors its order read-only for the startup line and the security check. - otelbridge/bridge.py:207 (signal ownership) — FIXED. `loop.add_signal_handler` replaces rather than chains, and asyncio exposes no way to restore the displaced callback, so the bridge no longer touches a signal the running loop already handles; `install_signal_handlers=False` turns registration off entirely for embedders. - otelbridge/mapper.py:611 (unbounded run.pipes) — FIXED. MAX_TRACKED_RUNS did not bound pipes: a run that keeps receiving events is re-inserted as most-recently-eventful and never evicted, while every begin (or event for a never-begun pipe) adds a _PipeState that only a matching end removes, and a reconnect does not replay missed ends. MAX_TRACKED_PIPES_PER_RUN now closes the least-recently-eventful pipe as unclosed, with recency updated on pipe activity rather than inferred from insertion order. - otelbridge/setup.py:202 (tracer leak) — FIXED. A failure anywhere in the metric half left the BatchSpanProcessor worker running with no handle to stop it, because `shutdown` had not been created yet. That path now shuts the tracer provider down before re-raising. Tests: 32 added (131 in tests/test_otel_*.py, was 99), covering stdout redaction, the refusal and its opt-out, env-var and signal-specific endpoint coverage, the non-redirecting session, the tracer teardown, signal ownership and the pipe cap's LRU behaviour. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(sdk): skip the loop signal-handler ownership test on Windows asyncio's ProactorEventLoop raises NotImplementedError from add_signal_handler, so the test cannot set up its precondition there. The bridge itself already tolerates that platform (the other signal tests pass on Windows); only the test needs the platform guard. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(cli): document the otel --insecure flag in the site CLI reference The `otel` command grew an `--insecure` opt-out for the cleartext-credential guard, but the docs-site "Key flags" table was never updated to match. Users hitting exit code 2 from `rocketride otel --endpoint http://tempo:4318 --headers Authorization=...` had no way to learn why from the CLI reference — the flag was only documented in the client-python OTel bridge guide. Add the missing row, naming the `ROCKETRIDE_OTEL_ALLOW_INSECURE` env var alongside the flag so the site table matches otel-bridge.md and the argparse help text. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sdk): declare the otel test dependency and stop orphaning bridge providers Two review findings on the `rocketride otel` bridge. 1. The opentelemetry dependency of the bridge tests was inherited, not declared. tests/test_otel_mapper.py and tests/test_otel_setup.py opened with pytest.importorskip('opentelemetry'), and nothing in the test path installs the 'otel' extra — the packages reached the engine interpreter only as a transitive dependency of the crewai node. Dropping or repinning that node would have turned ~1450 lines of mapper/setup coverage into a silent skip with CI still green. CI never pip-installs the package (pytest runs on the engine interpreter importing the package from the source checkout via tests/conftest.py), so the extras alone cannot fix this. The deps are added to packages/server/build-requirements.txt, which server:setup-test-deps installs through depends() under the merged constraints — the one install step that path goes through — and the 'dev'/'test' extras now pull 'rocketride[otel]' so a source checkout matches. Both importorskip guards become hard imports: a missing dependency now fails collection instead of skipping. 2. build_providers() always built both provider halves and adopted its shutdown only when shutdown_fn was None, so an embedder passing its own shutdown_fn left a TracerProvider and its BatchSpanProcessor worker thread with nothing to stop them — and the half the caller had already supplied via a factory was built, wired and discarded. build_providers() takes with_tracer / with_meter (defaulting to the old behaviour) and run_bridge asks only for the half it is missing; a caller-supplied shutdown_fn is now chained in front of the providers' shutdown — caller first, providers in a finally — instead of replacing it. The CLI path passes neither argument and is unaffected. Regression tests cover chain order, the provider shutdown surviving a raising caller, and which half is built for each factory combination; test_otel_bridge.py stays runnable without the extra. The embedded-use ownership rules are documented in the bridge doc. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * build(client-python): install the otel test deps in the test step, not the engine Review follow-up: the previous commit declared opentelemetry-sdk and opentelemetry-exporter-otlp-proto-http in packages/server/build-requirements.txt. That file is installed by server:setup-test-deps on every engine build, so every engine install carried the OTel SDK and the OTLP exporter whether or not anyone ran the bridge — which made the summary's "zero new base dependencies" true of the package but not of the engine. Move the cost to the step that needs it. A new client-python:setup-test-deps action installs packages/client-python/tests/requirements.txt into the engine interpreter through depends(), and runs only inside client-python:test, between the parallel build group and the test-server bracket: server:build -> [nodes:build | ai:build | client-python:build] -> client-python:setup-test-deps -> py-test-server:setup -> client-python:run-pytest -> teardown That ordering puts it after the node/ai requirement files land in the dist, so depends() resolves against the complete constraint set, and before the bracket, so a cold install does not hold the test server open. This mirrors the existing packages/ai/tests/requirements.txt pattern (packages/ai/scripts/tasks.js). packages/server/build-requirements.txt is restored byte-for-byte to its previous state. The 'rocketride[otel]' self-reference in the dev/test extras stays — that is the source-checkout path — and the hard imports in test_otel_mapper.py / test_otel_setup.py stay, so a missing dependency still fails collection instead of skipping. The tests no longer depend on the crewai node pulling opentelemetry in transitively either way. Docstrings in both test modules and the extras comment in pyproject.toml now point at the new file and step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(client-python): surface the OTel bridge on the Python SDK overview Every other page under docs/public/python/ has a card in the SDK overview's "Explore the SDK" grid; the newly re-homed otel-bridge page did not, so it was reachable only from the sidebar and the CLI reference. Adds the card last in the grid, matching sidebar_position: 13. Uses LuRadar, which is not already taken by another card in this file and is present in the Lucide set react-icons v5 ships. This replaces the "OpenTelemetry export (`rocketride otel`)" section that used to live in docs/README-python-client.md and packages/client-python/docs/index.md — both deleted by the docs revamp (rocketride-org#2139). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(clients): declare otel-bridge as a Python-only site page The client-docs gate requires both folders to publish the same page set, with single-language extras declared in two places: PY_ONLY_PAGES/TS_ONLY_PAGES in scripts/validate-client-docs.py, and the Site page set section of the schema. docs/public/python/otel-bridge.md was added without either, so 'Docs site build' failed with: python: undeclared site page otel-bridge.md. The page is genuinely Python-only — the TypeScript SDK ships no OTel bridge (no otel surface under packages/client-typescript/src, no page in docs/public/typescript/) — so it belongs in PY_ONLY_PAGES rather than being mirrored. Declared at sidebar_position 13, matching the TypeScript-only database-sequelize extra. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(sdk): name the private SDK attribute the OTel bridge hooks The bridge receives monitor events by assigning its dispatcher to RocketRideClient._caller_on_event — the attribute EventMixin.__init__ creates from the public on_event= kwarg and EventMixin.on_event awaits for every DAP envelope. It assigns rather than passing on_event= because it is handed an already-built client and must restore the previous handler on exit. Nothing in the SDK's public surface pins that name, and a rename would fail silently: assigning an unknown attribute to a Python object succeeds, so the bridge would keep connecting, keep subscribing, keep logging "reconnected", and never export another span. Lift the string into CLIENT_EVENT_HOOK with the rationale next to it, so the coupling is a single named seam a test can assert on instead of a literal repeated across three call sites. No behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(sdk): pin the OTel bridge against the real RocketRideClient The otel suite was mock-only past the mapper: test_otel_bridge.py and test_otel_cli.py each define their own FakeClient, and the provider tests monkeypatch build_providers, FlowSpanMapper and MetricsMapper out. That covers the run loop's branches but leaves the bridge's one point of coupling to the SDK untested — and a fake that defines _caller_on_event itself can never notice the SDK dropping it. Measured: renaming _caller_on_event to _caller_on_event_renamed across mixins/events.py and mixins/data.py — a change that stops the bridge exporting anything at all — leaves all 141 existing otel tests green. Add tests/test_otel_client_contract.py, which constructs a REAL RocketRideClient and stubs only the transport (call/is_connected); the SDK's add_monitor bookkeeping, EventMixin.on_event dispatch, the mappers and the OpenTelemetry provider pipeline all run for real: - three tests pinning CLIENT_EVENT_HOOK from different angles: the attribute exists in vars() on a real client, it is where the public on_event= kwarg lands, and the SDK's dispatch actually awaits it - run_bridge end to end on a real client, with events pushed through EventMixin.on_event rather than a fake's emit(), asserting routing, the unconsumed-event case, and hook restoration on shutdown - the wildcard subscription as the real SDK encodes it on the wire, plus the _monitor_keys entry the bridge relies on for replay after a reconnect - the 24-record wire fixture replayed to real exported spans, matching the 10-span forest test_otel_mapper.py gets by feeding the mappers directly, so the bridge provably adds, drops and reorders nothing Under the same simulated rename, six of these seven fail. Also add an opt-in live export against a real OTLP collector, gated on ROCKETRIDE_OTEL_TEST_ENDPOINT via the same skipif shape RocketRideClient_test.py uses for requires_llm. It starts no collector and changes no CI step; without the env var it simply skips. Note the new file in the client-python:setup-test-deps comment (it imports the OTel SDK at collection time) and cross-reference it from test_otel_bridge.py's docstring. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: nihalnihalani <24360630+nihalnihalani@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Run logging & replay: the task-event continuum, DVR sessions, and permanent trace identity
Companion saas PR rocketride-ai/rocketride-saas#360 advances the submodule and hosts the UI; merge this PR first, then the saas PR.
What this is
Today nothing survives a completed task: the engine child's stdout IS the event stream (consumed live, never persisted), and tasks are garbage-collected after a grace period. This branch makes every task write one continuous JSONL event log per
projectId.source.runKind— a continuum in which individual runs are just chapter markers — served back through a singlerrext_logcommand, replayed through the existing monitoring panels with a per-source DVR player, and consumed everywhere (live and replay alike) through one client session abstraction. Dev and deployed runs are naturally separate continua.Architecture, bottom up
L1 — every event carries its identity (
9bf99944, hardened in5fb2ca48,44a5c680; protocol-corrected in the DAP-fix commit)body.eventTime(epoch-seconds float, stamped once where raw stdout frames become JSON) andbody.logSeq— the continuum sequence, catalog-seeded (a fresh stream starts at 1; reopen continues atcontrol.lastSeq + 1), strictly monotonic across runs and engine restarts. logSeq is the ordering axis (immune to clock adjustment); time is the human axis.seqstays each connection's own monotonic message counter, exactly as DAP requires — an earlier iteration rode the continuum in the headerseq, which broke DAP sequencing the moment one connection monitored two tasks (event Y followed by event Y−555). The generic DAP handler (dap_conn.py) is byte-identical to develop: the envelope is transport, the body is the record.Task.build_eventassigns logSeq at build time (one seq per delivered event, no gaps from built-but-unsent events); the monitor forward is plainsend_event(event, id, body)— the shared body carries the stamps to every subscriber identically._forward_task_eventstampsproject_id+sourceinto every task-scoped body centrally, replacing per-event-builder stamping and client-side envelope-id suffix guessing. Why: membership must be a pure body test; identity established at one choke point cannot drift per event type.body.eventTime/body.logSeqdirectly — the SDKLogEvent(TS + Python) and shared-ui'sTaskEventMessagehave no top-level stamp fields; the body is typed with the stamps required. Legacy segments that carried the stamps at the header are canonicalized into the body at decode (normalizeStampsin both SDK codecs), so old recordings read identically to new and no consumer carries a fallback. The host gates admit an event on one pure body test and pass the message through untouched../builder shell:freeze(sanctioned: nothing has been released, so no consumer ever saw the pre-fixLogEventshape). The frozen contract now carries the body-only event type.L2 — the writer (
7feda2a2,44a5c680)RunLogWriterspools JSONL locally and seals immutable ~16MB segments (24h backstop) uploaded through the user-scoped account FileStore; a fixed 64-slot control file is the time/seq → segment routing table plus the chapters list (run begin/end intervals) that draws the activity bar without touching segments. No compaction, no per-run artifacts, no tokens anywhere in paths or content (tokens are credentials, not names).body.final=true) — clients never infer process death.rocketride/_log_codec.py,client/log-codec.ts— not exported).L3 — one read path (
b3086c4c)rrext_logmixin:chapters(the control file's run intervals with begin/end times + starting seq),read(ranged seq/time reads composing store + spool + in-memory tail, paging cursors, horizon truncation reported honestly),delete,segment(raw fetch for the session codec). Clients never touch storage.client.log.*namespaces together, with co-located docs.DVR sessions — the centerpiece (
f0f55c1e)client.log.openEventStreamreturns a LogEventStream session (TSclient/log-stream.ts, Pythonrocketride/log_stream.py, mirrored API):seek(pos|'live'),play(pos, speed, cb)(speed 0 = flat-out),pause,getStatus,getTraces,getTrace(beginSeq),getConsole,ingestLive,closeEventStream.Trace identity — a contract, not a heuristic (
f0f55c1e)A trace IS its begin event's continuum seq. Pipe slot ids recycle within a run; client fold docIds are synthetic counters — neither survives a refresh and both collide across runs. The begin seq is globally unique for the stream's life, so
getTrace(beginSeq)is deterministic from any position, live or replay: binary-search the span table to the owning segment, collect the slot's flow events forward to itsendacross segments and the live bucket, extend-on-miss for a still-growing tail segment. SSE narration (apaevt_ssebypipe_id) is collected into the same window, and the writer records which segments an in-flight document's SSE touched so a future sparse walk cannot skip SSE-only segments.UI (
6fca0391,82d4c88d,44a5c680,00a03bc6,f0f55c1e)SourceSection(own pill bar, own panes, own player) bound to its own continuum — sources replay independently, no shared cursor.useTaskEventsconsumes a session: append-only fold with runway throttling (the session plays flat-out, the hook pauses it past cursor + lookahead), fold caps reported honestly. Dead zone = cleared: outside any chapter every pane reports empty/zero at any position — stale panels in the gap between runs read as data from a run that never happened.runActive), not timeline-completion guesses, with a square growing edge.|</>|land on chapter boundaries with an epsilon so a boundary hit resolves into the intended track.apaevt_taskbegin announcement carriesbeginSeq— the run-begin marker's seq — so clients synthesize the chapter locally and reconcile with the server list by exact equality. A fetch-on-task-begin is both a race (control file not yet rewritten) and a thundering herd (every watching client at once).beginSeq, rendering the full call tree with Data|JSON|Raw and interleaved SSE narration rows. Analyze's slowest-documents rows open this popup — inspecting why a document was slow is the point of the table.>DBGBEGIN/ENTER/LEAVE/END on the real pipe id renders tool lanes (input/output/duration) in the trace tree.Riding along
2ad484a4— shell-api v0 recreated with the SDK client frozen in-bundle, append-only (prerequisite for the shared session surface).963e86ef— task arguments + debug output move from per-connection engine process flags to per-task pipeline settings applied via.use: they now work identically on local/on-prem/cloud and take effect on the next run, not the next engine restart.00a03bc6/87b15c07— SourceSection on the stock Card, the bespoke micro-button tier deleted, and its sizing returned as the stockButton miniprop aftersmallproved too large for canvas node cards.Testing
test_log_stream.pyagainst real segment fixtures:play()==read()equivalence, splice with no gap/no dup, console/status exact at position,getTraceacross recycled slots and segment boundaries, SSE inclusion, active-segment growth, direct live delivery, raced-fetch delivery, the 24MB split, LRU cap.Notes for reviewers
getTracewalk, vscode host migration to the session model, streamed chapter downloads, per-registrant monitor routing.🤖 Generated with Claude Code
Summary by CodeRabbit
LogEventStreamexperience.