Skip to content

docs(architecture): clarify diagrams and explain observation, reconciliation and rendering - #954

Merged
Juliusolsson05 merged 5 commits into
mainfrom
docs/architecture-observation-pipeline
Sep 12, 2026
Merged

Juliusolsson05 merged 5 commits into
mainfrom
docs/architecture-observation-pipeline

Conversation

@Juliusolsson05

@Juliusolsson05 Juliusolsson05 commented Sep 12, 2026 •

Copy link
Copy Markdown
Owner

This PR combines two architecture-reference changes that touched the same sections: the diagram clarity revision (formerly #940) and the observation, reconciliation and feed-rendering chapter (formerly stacked as this PR). #940 is closed in favor of this one; its two commits are included unchanged.

Part 1 — Diagrams that answer reader questions (#937)

Several architecture diagrams repeated nearby inventories or compressed too many relationships into small labels. This part keeps the opening application map and gives each detailed view a specific reader question, scope and takeaway.

  • Reduces the original 42 diagrams to 26, keeping the technical tables and every implementation link. The plan records every retain/remove decision.
  • Uses concrete examples for session identities, orchestration, missed workflow updates and live/saved message ownership. Sequence diagrams focus on ordering, failure and retry consequences.
  • Applies consistent blue/gray/amber roles, dashed external boundaries, readable labels and standalone titles and keys. Mermaid sources stay editable, with committed SVG previews.
  • Documents the design principles and primary references in Appendix B. The renderer now requires question and scope metadata and rejects obsolete previews.

Part 2 — Terminal observation, reconciliation and feed rendering (#947)

The reference did not explain the part that makes Agent Code possible. Agent Code has no provider API client: it runs the native CLI (or OpenCode's own server or TUI), observes it from outside, and reconstructs one sanitized conversation from channels that disagree about time, identity and trust. This part documents that pipeline end to end, from PTY bytes to painted rows, and the headless packages that produce the observations.

Headless packages (5.2, 5.3)

  • Corrects the submodule table to seven packages at their pinned revisions; it previously omitted opencode-terminal-headless and cited an old codex-headless pin.
  • Adds the observation points around a native process and what each is trusted for.
  • Adds the responsibility split between package, runtime adapter, main process and renderer.
  • Documents the shared channel model: semantic, screen and committed channels, plus a semantic shadow channel that keeps screen-derived text out of the conversation.
  • Documents each package's module structure, sanitization rules and known gaps:
    • claude-code-headless: exact transcript tail, mitmproxy flow attribution, subagent/suggestion/sidecar demotion.
    • codex-headless: causal rollout attribution with path leases, prompt-input evidence, strict semantic lifecycle.
    • The two OpenCode runtimes: bus assembly versus a read-only database plus live sequencing.
    • agent-transcript-parser: the neutral model and projection, and where it is and is not on the live path.

Conversation reconstruction and rendering (8.3)

  • Why reconstruction exists: the three historical failure shapes and the one-owner invariant.
  • Trust per fact rather than per channel, and where the no-screen-prose rule is enforced.
  • Ingest: the channel-to-reducer map, per-provider turn-replacement policy and the ghost lifecycle.
  • The sanitized object with a worked JSON example, plus complete tables of owners, candidate ids, all 25 reasons and unknown dispositions.
  • Collectors, the ledger pass, the live ownership decision chain, provider suppression policy, the five ghost rules and the ordering law.
  • A tick-by-tick worked Claude turn.
  • The view bridge and provider dispatch.
  • The A→B→C→D evidence method as architecture: fp2 fingerprints, the paint-time observer and recorder sidecar, per-provider shapes.ts catalogs, classification, and the audit and coverage gates.
  • Fixtures, the replay harness and its five per-tick invariants, with the harness's honest limits.

Supporting sections

  • 8.2.1 lists every observation IPC channel with its producer, main-process policy and whether it is recorded.
  • The constraints and strategy sections now state the native-client constraint and the evidence-first approach.
  • Adds risk and change-map rows for the gaps found, plus glossary terms.

Findings documented rather than smoothed over:

  • Four declared RenderReason values and three owners are never produced.
  • Ledger decisions have no runtime reader outside replay and tests.
  • The recording corpus holds two hand-built fixtures, and the named fixture tests use hand-authored inputs.
  • The phone passes a zero committed tail, which disables the collapsed-running rule.
  • Fresh Codex attribution depends on the attested 0.149.1 input profile.
  • Structured OpenCode has replay and reconnect gaps, and its registered fold policy contradicts its type comment.

Part 2 adds 20 diagrams and replaces rendering-pipeline, for 45 in total. Inventories stay tables.

Validation

  • Links: the Markdown check found no broken local paths, anchors or package source paths.
    • Every package link uses the gitlink pinned at this revision.
    • Every RenderReason and RenderOwner in source appears in the document.
  • Part 1 previews: 26/26 diagrams parsed, rendered and reproduced byte for byte with --check in docs(architecture): make diagrams answer reader questions #940's environment. They were reviewed at 960px and in grayscale, and the tested palette pairs meet the 4.5:1 text and 3:1 line contrast targets.
  • Part 2 previews: all 20 new or edited previews reproduce byte for byte with --check and were inspected at full resolution for width, clipped labels and mid-word wrapping.
    • The 17 Part 1 previews that Part 2 does not touch do not reproduce in the second machine's Chrome/font environment. They are left exactly as rendered in Part 1 rather than churned.
  • Source verification: claims taken from source surveys were re-verified against source before being stated as fact.
  • Tests: no application code changed, so no application tests were run. A trial merge with current main is clean, and main has not touched the architecture document since the base.

Fixes #937
Fixes #947
Refs #100

🤖 Generated with Claude Code

https://claude.ai/code/session_01FcJNPX21SkzfKhRFxftriT

Juliusolsson05 and others added 5 commits September 12, 2026 14:05
Audit all 42 views, retain 26 focused explanations, and use consistent visual roles with accessible standalone previews. Preserve implementation links and document the research and visual validation.

Refs #937, #100
The architecture reference lists the headless packages as table rows and
compresses the ownership ledger into a short section, so a maintainer cannot
follow a native terminal session into the painted feed. Plan an exhaustive,
source-grounded chapter covering the native-client constraint, package
structure, channel trust, ingest, the sanitized ledger object, the shape
evidence loop and fixture replay.

Refs #947

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FcJNPX21SkzfKhRFxftriT
…feed rendering

The reference described most subsystems but not the one that makes Agent
Code possible: running native provider clients, observing them through
the terminal mirror, model stream, transcript files and process, and
reconstructing one sanitized conversation before anything paints.

Document the seven headless packages with their module structure,
channel model, sanitization rules and known gaps. Replace the short
ownership section with a source-grounded chapter covering channel trust,
ingest reducers, the sanitized ledger object, collectors, ownership and
ghost rules, the ordering law, a worked turn, provider dispatch, the
shape recorder and catalogs, and fixture replay with its invariants.
Add 20 diagrams and record the gaps found in the risks and change map.

Fixes #947

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FcJNPX21SkzfKhRFxftriT
Both changes rewrite the same reference document and diagram set, so they
are reviewed as one PR against main instead of a stack.

Refs #937, #947

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FcJNPX21SkzfKhRFxftriT
@Juliusolsson05 Juliusolsson05 changed the title docs(architecture): explain terminal observation, reconciliation and feed rendering docs(architecture): clarify diagrams and explain observation, reconciliation and rendering Sep 12, 2026
@Juliusolsson05
Juliusolsson05 changed the base branch from docs/architecture-diagram-clarity to main September 12, 2026 22:28
@Juliusolsson05
Juliusolsson05 marked this pull request as ready for review September 12, 2026 22:36
@Juliusolsson05
Juliusolsson05 merged commit c87448a into main Sep 12, 2026
2 checks passed
@Juliusolsson05
Juliusolsson05 deleted the docs/architecture-observation-pipeline branch September 12, 2026 22:40
Juliusolsson05 added a commit that referenced this pull request Sep 20, 2026
Main gained three quit-time owners after this branch was cut, and each
had to be placed in the committed-shutdown composition rather than left
on before-quit, which runs before the renderer's unload vote:

- The extension runtime (#577). Its PAUSE stays on before-quit: it is
  the one preparation that must finish before windows unload, and it is
  reversible (Keep Editing resumes it). Its disposal is a new committed
  stage, `stopExtensions`, which runs beside sessions and workflows —
  extensions run user code — and only AFTER startup settles, because
  startup creates the runtime and an earlier stop would complete against
  nothing and keep that receipt.
- The performance monitor and trace drain (#958), with its 1.8s/2s
  bounded race, folded into the `performance` stage. Main also ran
  monitor/watchdog/performanceService stops in will-quit; those are
  already stages here, so the duplicates are dropped.
- tmux recovery now goes through restoration's canonical envelope
  decoder. A quit committed mid-read must not authorize killing tmux
  sessions, so the startup assertion moved inside the reader, where it
  becomes the decoder's read-failure path and withholds cleanup.

`workflowShutdownPromise`/`Complete` are gone with the before-quit
handler that used them; workflow stop is a committed stage.

The shutdown diagram was rewritten as a flowchart in main's current
convention (accTitle/accDescr/scope, #954) and says what the composition
actually does: the editor decides first and Keep Editing leaves every
service live; only a committed quit stops execution; a failed stop keeps
the app and its process lock. Re-rendered with the pinned toolchain;
every other SVG restored.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs(architecture): explain terminal observation, reconciliation and feed rendering docs(architecture): make diagrams explain useful relationships

1 participant