Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 82 additions & 0 deletions devlog/_plan/261013_deferred_v1_collab_surface/000_plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Deferred V1 collaboration surface (#6844 remainder) and #6984 drain deadline — lane L3

Release train 2.84.0, lane L3, branch `codex/rt284-issues-proxy`, first cut from `origin/dev` `3e58c2ce3e`
and later rebased onto lane L1's restacked head `1e4d9dad9a` (stack L2 → L1 → L3 → L4).
Coordinator thread `01a130be-2e10-7af0-adf9-efca9fe37e30` owns push, PR, merge and release.

## Objective

1. #6844 remainder. PR #6977 made `collabSurface()` read V1 tools that codex-rs renders inside the
code-mode `exec` description. When the model supports tool search (every OpenCodex catalog row),
codex-rs registers V1 as **deferred** and renders none of it, so those turns stay `null` and never
receive the V1 Max/Ultra proactive guidance. Add a second, positive-only evidence source: the
developer `<tools>` world-state fragment that lists deferred tool namespaces.
2. #6984. Verify that owner PR #6986 (lane L1, "Closes #6984") fixes the picker relay on Bun 1.4.2 and
1.4.3 and record the residual. As first planned, no listener change in this lane: a parallel edit of
`src/claude/intercept/picker-listener.ts` would conflict with L1's stack. **Amended:** the coordinator
then approved a stacked commit on L1 that bounds the post-EOF drain #6986 added; see
`020_6984_verification.md`.

## Evidence (codex-rs `openai/codex` @ `16845bb8e8`, 2026-10-12)

- `core/src/context/world_state/tools.rs:29-147` — `ToolsState` renders a **developer** fragment
`<tools>\nDeferred tool namespaces:\n- <ns>[: <first line, <=250 chars>]\n...</tools>` the first time,
then only diffs `Added deferred tool namespaces:` / `Removed deferred tool namespaces:`, and
`No deferred tool namespaces remain.` when empty (`:159`). Omitted rows render
`... <n> additional namespaces omitted.` (`:203-210`); the fragment is capped at 4 KiB (`:25`);
names are reserved before descriptions (`tools_budget.rs:27-69`), so a row may be name-only.
- `core/src/session/world_state.rs:318-326` adds the section only when `Feature::DeferredToolWorldState`
is enabled; `features/src/lib.rs:1572-1575` marks it UnderDevelopment, default off. Codex Desktop
sessions routed through OpenCodex were observed receiving it (`- multi_agent_v1: Tools for spawning and
managing sub-agents.`). Feature-off clients stay undetectable — documented, not claimed fixed.
- `core/src/tools/spec_plan.rs:1561-1588` registers V1 tools with `ToolExposure::Deferred` when
`supports_search_tool`; `core/src/tools/handlers/multi_agents_spec.rs:14` fixes the namespace
`multi_agent_v1`. V2 tools are `Direct`/`DirectModelOnly` (`spec_plan.rs:1468-1472`), never deferred.
- OpenCodex: developer items reach `parsed.context.messages` with role `developer`
(`src/responses/parser.ts:318-330`); no OpenCodex code injects or strips `<tools>` items.
Structured `tool_search` tools carry `toolSearch: true` (`src/responses/parser-tools.ts:120-128`).

## Architect consultation (gpt-6.1-sol, agent 01a130cb-020d-78e3-bd33-396cb4d12bfb)

| ID | Proposal | Main disposition |
|---|---|---|
| D1 | New `deferred-collab-surface.ts`, read developer messages only, standalone `<tools>` fragments | Accepted |
| D2 | Ordered replay: snapshot replaces, added adds, removed removes, empty clears; omitted removals invalidate carried membership; omitted additions never establish; malformed fragment clears carried evidence | Accepted |
| D3 | Host coherence: genuine code-mode exec (existing guards) or a structured `toolSearch` tool; expose rendered foreign-collab claim even without rendered V1 | Accepted |
| D4 | Combination in `collabSurface()` mirrors #6977; deferred evidence never yields V2 | Accepted |
| D5 | Docs: `structure/subagents.md`, English `sub-agent-surface` guide, stale helper comments | Accepted; commit says "Refs #6844", closure is the coordinator's decision because feature-off clients remain undetectable |

Architect for #6984 (agent 01a130cb-02bd-77d2-9f9b-2feaac02de82) proposed a raw-TLS h1 transport
rewrite (D1-D6). Disposition: **rejected for this lane** — #6986 already lands a targeted listener fix
(its PR reports green on Bun 1.4.0/1.4.2/1.4.3; this lane re-ran 1.4.2 and 1.4.3); the rewrite is a
separate design. Residual recorded: the post-EOF ciphertext drain added by #6986 had no deadline —
the coordinator approved bounding it in this lane (architect D4's drain budget, without the rewrite).

## Work-phase map

| Phase | Doc | Outcome |
|---|---|---|
| wp1 | `010_deferred_v1_evidence.md` | Commit `33747c4188` (rebased from reviewed `86ea92a69d`): deferred V1 evidence + tests + docs |
| wp2 | `020_6984_verification.md` | Verification of #6986, then commit `5e943bfe69`: bounded post-EOF discard + deterministic test |
| wp3 | this record | Gates, independent sol review of final head, status report, this docs commit |

## Out of scope

Push, PR, merge, issue comments and issue closure (coordinator-owned), other lanes' files.
`picker-listener.ts` was out of scope until the coordinator approved the stacked drain deadline.

## Final state

- Head of code commits `5e943bfe69ccfc830937374bf8ccd443871407d1` on L1 `1e4d9dad9ab8c133b0f7c531ed52734d65b9b2d8`;
this docs commit follows it. Nothing pushed.
- `33747c4188` and the reviewed `86ea92a69d` share `git patch-id --stable` `34db79974f`; the final
review confirmed every non-layout blob byte-identical and both layout maps hold every entry from
both sides (`scripts/test-layout/layout.json` 1999 lines).
- Reviews (gpt-6.1-sol): #6844 architect `01a130cb-020d` ALIGNED after two folds; plan audit
`01a130d2-b434` FAIL → NEAR-PASS; #6844 final-head review `01a130e2-226d` FAIL (U+2028/U+2029,
repeated headings) → PASS at `86ea92a69d`; stacked final-head review `01a130f4-959e` NEAR-PASS
(P3 doc overstatement) → PASS at `5e943bfe69`; wp3 plan audit PASS.
- Lane terminal status: both lane commits done and reviewed. Pushing, the stacked PR, merge, release
and closing #6844/#6984 belong to the coordinator. #6844 stays "Refs": feature-off Codex clients
remain undetectable.

Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# 010 — Deferred V1 evidence from the `<tools>` world-state fragment

## Files

- NEW `src/server/responses/deferred-collab-surface.ts`
- MODIFY `src/server/responses/code-mode-collab-surface.ts` — export `analyzeCodeModeExec(tools)`
returning `{ genuineExec: boolean; renderedV1: boolean; foreignCollabClaim: boolean }`;
`codeModeV1Evidence()` becomes a wrapper with unchanged results. Update the header comment that says
deferred V1 has no evidence.
- MODIFY `src/server/responses/collaboration.ts` `collabSurface()` (`:319-327`).
- NEW `tests/codex-integration/deferred-collab-surface.test.ts`; register it by appending to an
existing line of `scripts/test-layout/layout.json` (file stays at 1999 lines) and in
`tests/fixtures/test-layout-expected.json`.
- MODIFY `structure/subagents.md` (detection paragraph), `docs-site/src/content/docs/guides/sub-agent-surface.md`
(v1 code-mode sentence).

## deferred-collab-surface.ts

```ts
export function deferredV1Evidence(parsed: OcxParsedRequest): boolean
```

1. Host coherence first: true only if `analyzeCodeModeExec(tools).genuineExec` or some un-namespaced
tool has `toolSearch === true`. Otherwise false.
2. Walk `parsed.context.messages` in order; only `role === "developer"`. For string content use the
whole string as one candidate; for array content, each `type === "text"` part separately.
3. A candidate is considered only if, after trimming surrounding whitespace, it starts with `<tools>`.
Anything else is ignored: prose, quotes and fences that merely mention the tag never qualify.
A considered candidate that does not end with `</tools>` is a broken envelope and counts as
**malformed** (clears carried evidence). The body between the outer tags is validated only by the
line grammar in step 4: a row's description is opaque (upstream copies the first description line
unescaped, `world_state/tools.rs:54`, `tools_budget.rs:32`), so `<tools>`/`</tools>` inside a
description is fine, while a stray envelope on its own line fails the grammar and is malformed.
4. Parse the body line by line (`\n`, ignore empty lines):
- group headers `Deferred tool namespaces:` (snapshot), `Added deferred tool namespaces:`,
`Removed deferred tool namespaces:`;
- rows `- <ns>` or `- <ns>: <text>` under the current header, `<ns>` = `[^:\s]+`, `<text>` opaque;
- omission `... <n> additional namespaces omitted.` under the current header;
- `No deferred tool namespaces remain.`;
- any other line, a row before a header, snapshot mixed with added/removed, or a namespace listed
under both Added and Removed in the same fragment => malformed.
5. Apply in order to `current: Set<string>` plus `known: boolean`:
- malformed: clear, `known=false`;
- snapshot: `current = rows` (omitted rows unknown, not inherited);
- diff, in this order: (a) an omission under Removed clears `current`; (b) add the explicit Added
rows; (c) delete the explicit Removed rows. An omission under Added adds nothing;
- empty-state line: clear.
6. Result: `current.has("multi_agent_v1")`.

### Capability boundary (audit finding 1, disposition: rebutted with evidence, documented)

The namespace row proves that at least one deferred `multi_agent_v1` tool is registered, not that
`spawn_agent` is. Upstream can only split them through an embedder-supplied `ToolPolicy.allowed_tools`
ceiling (`ext/extension-api/src/tool_policy.rs:10-44`, applied per tool in `core/src/tools/registry.rs:428`).
The only in-tree production allowlist, `reviewer_tool_policy` (`ext/guardian-reviewer/src/settings.rs:44-55`),
admits no `multi_agent_v1` tool, so the whole namespace disappears; the V1 depth limit disables the surface
as a unit (`core/src/tools/spec_plan.rs:728-740`). Codex CLI/Desktop therefore never emit a
management-only V1 namespace. A third-party embedder with a custom allowlist could; that case receives
advisory guidance it cannot act on. This is the accepted fallback for #6844 and is stated in
`structure/subagents.md` and the helper header; a test pins the namespace-only classification so a
future change to it is deliberate.

## collabSurface()

```ts
export function collabSurface(parsed: OcxParsedRequest): "v1" | "v2" | null {
const direct = directCollabSurface(parsed.context.tools);
const codeMode = analyzeCodeModeExec(parsed.context.tools);
if (codeMode.renderedV1 && codeMode.foreignCollabClaim) return null; // #6977 ambiguous
const inferredV1 = codeMode.renderedV1 || deferredV1Evidence(parsed);
if (!inferredV1) return direct.surface; // unchanged path
if (codeMode.foreignCollabClaim) return null; // deferred V1 vs rendered V2
return !direct.sawNativeCollabTool || direct.surface === "v1" ? "v1" : null;
}
```

Rendered-only behaviour is identical to #6977 (rendered+foreign → null; rendered alone → same rule).

## Tests (`tests/codex-integration/deferred-collab-surface.test.ts`, through `parseRequest`)

Positive: snapshot + genuine exec → v1 and Max/Ultra guidance text; structured tool_search host → v1;
additional_tools envelope; name-only row; descriptions containing `<tools>` and `</tools>` keep the
fragment valid; namespace-only classification pinned (documented capability boundary).
Gates: medium and disabled stay silent.
Replay: snapshot replaces; add-remove-add; addition without baseline; empty state clears; full
snapshot without V1 drops prior V1.
Budget: omitted snapshot rows do not establish; omitted additions do not; omitted removals invalidate
carried V1; explicit addition in that same diff survives; a diff listing `multi_agent_v1` under both
Added and Removed (with and without omitted removals) is malformed and clears evidence.
False positives: user/assistant text quoting the block; developer text embedding it mid-prose; lookalike
namespace `multi_agent_v1x`; caller function named `tool_search` without the structured marker and
no exec; malformed fragment clears carried evidence; valid V1 snapshot followed by an unterminated
`<tools>` fragment yields no deferred evidence; a stray `<tools>` line inside a fragment body is
malformed. Host guards repeated **with an otherwise valid V1 block**: exec declared as a function,
namespaced exec, exec beside a flat shell bridge, exec whose description lacks the codex-rs template
line → all null.

Continuation (`rememberResponseState` + `expandPreviousResponseInput` + `parseRequest`): the suite sets a
temporary `OPENCODEX_HOME` before any state access, resets state inside that sandbox, settles pending
persistence before cleanup and restores the environment afterwards, following
`tests/responses/responses-state.test.ts:279`. Chain: stored snapshot → continuation with no new fragment keeps v1; → continuation carrying a
Removed row → null; → later continuation carrying an Added row → v1 again; → continuation carrying
`No deferred tool namespaces remain.` → null.
Combination: direct V1 agrees; direct V2 conflicts → null; rendered V2 listing + deferred V1 → null;
MCP collaboration-named tools keep existing rules; deferred evidence never yields v2.

## Verification

`bun test` (Bun 1.4.3) on the new file, `tests/codex-integration/code-mode-collab-surface.test.ts`,
`multi-agent-compat.test.ts`, `effort-policy.test.ts`; `bun run typecheck`; layout, tooling and
file-size guards; `bun run structure:check`; `bun run privacy:scan`.

Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# 020 — #6984: verify #6986, then bound its post-EOF drain

## 1. Verification of #6986 (lane L1)

#6986 (owner PR, in L1's bundle as `1c1e6c994d`) states "Closes #6984" and patches
`src/claude/intercept/picker-listener.ts`: a closing h1 reply replaces the socket's `destroySoon` with
resume + graceful `end()`, the TCP front sets `allowHalfOpen`, and on backend `end` unpipes and resumes
the client so the abandoned upload keeps draining.

Baseline at `origin/dev` `3e58c2ce3e`, `tests/claude-integration/claude-picker-upload.test.ts -t refusal`
(`OCX_TEST_NO_QUEUE=1 <bun> test --isolate ...`): Bun 1.4.0 8/0; Bun 1.4.3 6/2 (buffered h1 400/503:
`read ECONNRESET`); Bun 1.4.2 (darwin) buffered h1 503 hits the 4 s test deadline.

At #6986 head `ad1540aa43` (scratch checkout): the full upload file passed 22/0 in three runs each on
Bun 1.4.2 and 1.4.3.

## 2. Coordinator decision

First plan: no listener change here (it would conflict with L1). The coordinator then approved a
stacked commit on L1's restacked head `1e4d9dad9a`: keep reply delivery and `Connection: close`, stop a
trickling peer from holding the socket indefinitely, keep `claude-picker-upload` green on Bun 1.4.2 and
1.4.3, and add a deterministic regression test (fake timers or an injected clock, no timeout increases).

## 3. Change — commit `5e943bfe69` ("Refs #6984")

- `PICKER_ABANDONED_INPUT_LINGER_MS = 30_000`. After the backend ends a spliced connection, the front
still unpipes and discards client input, and once the reply has been flushed to the client socket
(`finish`) it arms a drop after that budget; the connection's `close` (including listener shutdown,
which destroys tracked sockets) cancels it.
- Limits: `finish` is local flush, not peer receipt — a peer that also stops reading for the whole
budget can lose unread reply bytes when the drop fires (verified by the reviewer with an immediately
fired injected drop and a paused reader: `ECONNRESET`, 0/131,072 body bytes). A reply whose flush
stalls never reaches `finish`, so the budget is not an absolute connection-lifetime deadline.
- Test seam `scheduleAbandonedInputDrop(drop, ms) => cancel`; production passes none and uses a timer.
- `tests/claude-integration/claude-picker-linger.test.ts`: a half-open TLS client (`allowHalfOpen: true`)
that never stops uploading gets the 400 with `Connection: close`; the drop is armed exactly once
with the production budget; the connection is still open after 64 more flushed chunks; firing the
drop closes it; a client that disconnects cancels the armed drop. No wall-clock thresholds.
Mutation (arming removed): 0 pass / 2 fail.
- Bun 1.4.0: an instrumented run (`L3-linger-debug.ts`, lane scratch) showed the front socket closing
about 0.2 ms after the reply, cancelling the drop, so the "still open" assertion does not hold there
and the new tests were not claimed to pass on 1.4.0. The repository pins `testRunnerBun` 1.4.3.
- `structure/clients/claude-desktop.md` transport section updated (request-side drain still has no
deadline; the front-side discard is bounded, with the limits above).

## 4. Verification at `5e943bfe69`

`/private/tmp/rt284-lanes/status/L3-gates2.sh` (exit 0; recorded with `cxc receipt test`):
`bun run typecheck` 0; Bun 1.4.2 `claude-picker-upload` + `claude-picker-linger` 24 pass / 0 fail;
Bun 1.4.3 same files 24/0; Bun 1.4.3 deferred/code-mode collab, multi-agent compat, effort policy,
combo-failover e2e 366/0; Bun 1.4.3 test-layout, test-layout tooling (pinned runner), file-size
ratchet, repo hygiene, structure SSOT, skill-ocx 110/0; `bun run structure:check` 0;
`bun run privacy:scan` 0; `layout.json` 1999 lines. The same gate script, run again at the docs commit
that follows, also exited 0. Independent reviewer evidence is from `465207459c`, the head before the
wording-only amend that produced `5e943bfe69`: it re-ran upload + linger 24/0 on Bun 1.4.2 and 1.4.3,
151/0 with `OCX_TEST_NO_QUEUE=1 /Users/jun/.cache/rt284-bun/bun-darwin-aarch64/bun test --isolate
tests/codex-integration/deferred-collab-surface.test.ts tests/codex-integration/code-mode-collab-surface.test.ts
tests/claude-integration/claude-picker-listener.test.ts tests/test-layout.test.ts
tests/test-layout-tooling.test.ts tests/ci-workflows/file-size-ratchet.test.ts` (Bun 1.4.3), plus
typecheck, structure and privacy (exit 0). It then reviewed the `465207459c`→`5e943bfe69` delta as
wording-only and returned PASS without reruns.

`bun run test:changed` (dev merge base, so it covers L2 + L1 + L3; 1861 files): **non-green, 40,452
pass / 20 fail**. Failing cases: `tests/codex-integration/native-grok-toggle.test.ts` (GET reports absent
when no fence exists; a foreign-home install state refuses disable…; a second concurrent PUT gets 409
config_busy…; disable removes only the fence…; disable → enable → disable…; the ownership preflight
itself is reachable…; toggling to the current state changes nothing), `native-codex-toggle.test.ts`
(enable inspects the fixture's service-manager evidence; the status read follows an off-then-on round
trip…), `tests/service/service-ownership-state.test.ts` (claiming with no install state…; exhausted
revision refuses…), `tests/service/service-sqlite-home.test.ts` (four Codex SQLite home binding cases),
`tests/service/service.test.ts` (two service diagnostics staleness cases) and
`tests/cli/cli-access-audio-live.test.ts` (native open and non-readiness events…, three 5 s timeouts).
Isolation, `OCX_TEST_NO_QUEUE=1 <bun 1.4.3> test --isolate <file>` on both `5e943bfe69` and L1 `1e4d9dad9a`:
native-grok 24/0, native-codex 14/0, service-ownership-state 46/0, service-sqlite-home 5/0,
service 211/0, cli-access-audio-live 36/0 — identical on both heads. The combined run overlapped other
lanes' suites on the same machine; isolation passing does not prove the cause, and the combined-run
failures stay recorded for the coordinator's hosted CI.

Loading
Loading