Skip to content

Prototype opt-in managed context replacement for L2 reuse - #518

Draft
radianceded wants to merge 19 commits into
Gnosil:mainfrom
radianceded:codex/byte-replacement-runtime
Draft

radianceded wants to merge 19 commits into
Gnosil:mainfrom
radianceded:codex/byte-replacement-runtime

Conversation

@radianceded

@radianceded radianceded commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

When an admitted L2 Context slice duplicates an earlier managed JSON context message, strict injection sends that context twice. The new opt-in semantix.mode = "replace" compacts that original JSON message in the provider request and replaces the duplicate L2 body with a short reference retaining the original slice provenance. If the single-slice content, provenance, or unique earlier message does not match, it keeps the existing injection path. Session history remains unchanged.

The GLM-5.3-Flash Coding Plan comparison was rerun after aligning the managed message's project tag with real L2 provenance. In 3 controlled fact-extraction tasks (9 live calls), all answers were correct; replacement used 541 input / 867 total tokens versus injection's 1253 input / 1607 total. Raw usage and limitations are in docs/reports/byte-replacement/comparison-5.3-flash-plan-aligned.jsonl and result.md. The earlier Plan report had mismatched project tags. Both live reports build the L2 block in Go but construct the replacement request in Python; neither exercises the Go runtime branch. The focused Go check validates that branch separately. This does not establish production-wide answer equivalence or measured monetary/Plan-point savings.

Read-only inspection of 544 older local coding-session mirrors found no managed-context marker and 0/537 Context copies even when checking exact substrings anywhere in non-system messages from analyzable sessions. Although 394/404 Result bodies matched earlier assistant messages, that old format had no host verification fields, so none qualifies for the current verified-Result L2 gate. The existing memory_flow_e2e_test.go also shows strict L2 can carry historical facts absent from a new session. These findings keep the mode experimental: removing arbitrary L2 blocks would lose information, and no production-wide token or cost saving is established.

Experiment caveat: the nine-call report used an arm-specific prefix in the system message, so it is not a perfectly same-prompt comparison. The runner now keeps that message identical within each case; the first rerun call hit Coding Plan HTTP 429 / code 1302, and no revised comparison is available yet. The original token counts remain recorded, with this limitation in result.md.

The next comparison can use run_runtime.py: a local dry run captured all nine off/strict/replace requests through Agent.Run in isolated projects, with matching system/current messages and the expected real Go replacement branch. The first live GLM case returned the expected facts in all three arms; the remaining two cases and account charges are unmeasured after HTTP 429 / code 1302.

For the revised runtime behavior, the three dry-run requests total 6,081 UTF-8 content bytes in strict mode and 4,154 in replace mode (31.7% fewer). The earlier 56.8% input-token reduction came from the Python arm that removed L2 entirely and does not estimate this provenance-preserving branch. For the revised branch, one live case used 375 input / 516 total tokens versus strict injection at 592 input / 718 total, with both answers correct. Two cases and actual charges remain unmeasured.

When the shortened reference reaches the final provider request, delivery accounting now records its actual bytes and slice attribution; if an interceptor removes it, no delivery is credited. The focused agent check covers this path.

The same read-only census also ran the real slice.Distill path: 495 additional Context and 1,054 Memory cards had zero exact substring overlaps with source-session non-system messages. This supports keeping the current exact-match mode experimental; natural production C/M savings are not established.

Validation: go test ./harness/agent -run TestManagedContextReplacementKeepsTranscriptAndFallsBack -count=1; git diff --check. The focused check builds its block through the real L2 injector. No production source currently emits managed context automatically, so the mode remains opt-in and narrow.

Does not close #493 or its broader evaluation scope.

@radianceded
radianceded marked this pull request as draft September 28, 2026 04:51
@radianceded radianceded changed the title Add opt-in managed context replacement for L2 reuse Prototype opt-in managed context replacement for L2 reuse Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[研究] 以受控上下文字节替换减少 L2 重复输入

1 participant