Repository navigation
Prototype opt-in managed context replacement for L2 reuse - #518
Draft
radianceded wants to merge 19 commits into
Draft
radianceded wants to merge 19 commits into
radianceded wants to merge 19 commits into
Conversation
radianceded
marked this pull request as draft
September 28, 2026 04:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When an admitted L2 Context slice duplicates an earlier managed JSON context message, strict injection sends that context twice. The new opt-in
semantix.mode = "replace"compacts that original JSON message in the provider request and replaces the duplicate L2 body with a short reference retaining the original slice provenance. If the single-slice content, provenance, or unique earlier message does not match, it keeps the existing injection path. Session history remains unchanged.The GLM-5.3-Flash Coding Plan comparison was rerun after aligning the managed message's project tag with real L2 provenance. In 3 controlled fact-extraction tasks (9 live calls), all answers were correct; replacement used 541 input / 867 total tokens versus injection's 1253 input / 1607 total. Raw usage and limitations are in
docs/reports/byte-replacement/comparison-5.3-flash-plan-aligned.jsonlandresult.md. The earlier Plan report had mismatched project tags. Both live reports build the L2 block in Go but construct the replacement request in Python; neither exercises the Go runtime branch. The focused Go check validates that branch separately. This does not establish production-wide answer equivalence or measured monetary/Plan-point savings.Read-only inspection of 544 older local coding-session mirrors found no managed-context marker and 0/537 Context copies even when checking exact substrings anywhere in non-system messages from analyzable sessions. Although 394/404 Result bodies matched earlier assistant messages, that old format had no host verification fields, so none qualifies for the current verified-Result L2 gate. The existing
memory_flow_e2e_test.goalso shows strict L2 can carry historical facts absent from a new session. These findings keep the mode experimental: removing arbitrary L2 blocks would lose information, and no production-wide token or cost saving is established.Experiment caveat: the nine-call report used an arm-specific prefix in the system message, so it is not a perfectly same-prompt comparison. The runner now keeps that message identical within each case; the first rerun call hit Coding Plan HTTP 429 / code 1302, and no revised comparison is available yet. The original token counts remain recorded, with this limitation in
result.md.The next comparison can use
run_runtime.py: a local dry run captured all nine off/strict/replace requests throughAgent.Runin isolated projects, with matching system/current messages and the expected real Go replacement branch. The first live GLM case returned the expected facts in all three arms; the remaining two cases and account charges are unmeasured after HTTP 429 / code 1302.For the revised runtime behavior, the three dry-run requests total 6,081 UTF-8 content bytes in strict mode and 4,154 in replace mode (31.7% fewer). The earlier 56.8% input-token reduction came from the Python arm that removed L2 entirely and does not estimate this provenance-preserving branch. For the revised branch, one live case used 375 input / 516 total tokens versus strict injection at 592 input / 718 total, with both answers correct. Two cases and actual charges remain unmeasured.
When the shortened reference reaches the final provider request, delivery accounting now records its actual bytes and slice attribution; if an interceptor removes it, no delivery is credited. The focused agent check covers this path.
The same read-only census also ran the real
slice.Distillpath: 495 additional Context and 1,054 Memory cards had zero exact substring overlaps with source-session non-system messages. This supports keeping the current exact-match mode experimental; natural production C/M savings are not established.Validation:
go test ./harness/agent -run TestManagedContextReplacementKeepsTranscriptAndFallsBack -count=1;git diff --check. The focused check builds its block through the real L2 injector. No production source currently emits managed context automatically, so the mode remains opt-in and narrow.Does not close #493 or its broader evaluation scope.