Skip to content

docs: capture the whole-file read class and a guard-reachability check - #192

Merged
chattymin merged 1 commit into
mainfrom
docs/whole-file-read-class
Aug 19, 2026
Merged

chattymin merged 1 commit into
mainfrom
docs/whole-file-read-class

Conversation

@chattymin

Copy link
Copy Markdown
Owner

Summary

Two documentation changes, no source changes.

1. Record the whole-file read class (docs/reference/defect-log.md).
#184 moved Codex rollout parsing to a chunked FileHandle stream. The class behind
that fix lived only in the PR description, which stops being reachable once the PR
scrolls out of view. The entry captures:

  • why the whole-file read costs twice the file size, and why a per-line
    autoreleasepool cannot release the source string;
  • the two mechanics that make the streaming version correct and bounded — splitting
    on the 0x0A byte (safe because UTF-8 continuation bytes are all >= 0x80) and a
    per-chunk autorelease boundary;
  • a re-reading of perf: drain per-line autoreleasepool in usage-log parsing (peak -43%, steady -71%) #94: streaming was judged to regress memory because that chunk
    boundary was missing, so that judgement should not be cited to rule streaming out;
  • the providers that still read whole files (parseClaudeFile, parseGrokFile,
    parseGeminiFile) and why converting them is deliberately deferred;
  • that the regression guard is opt-in and therefore does not run in CI.

2. Add a reachability step to the defect protocol (CLAUDE.md).
Before adding or requesting a guard, confirm the scenario can occur in this codebase.
Injecting a defect proves a guard works; it does not prove the situation is reachable.
The worked example is the #181 review, where a test was requested for Codex root
folding and then withdrawn: there is no CODEX_HOME, so the roots are two hardcoded
siblings that cannot overlap, and the production callers are already normalized.

Type of change

  • Bug fix
  • New feature
  • Refactor / cleanup
  • Documentation

UI changes

None.

Checklist

  • PR title and description are written in English
  • No source changes — documentation only
  • No copyrighted assets, secrets, or private tooling references are committed
  • Every symbol cited in the new entry was verified to exist

Validation

  • Cited symbols confirmed present: forEachCodexLine, parseClaudeFile,
    parseGrokFile, parseGeminiFile, CodexLargeRolloutPerformanceTests,
    POKETOKENBAR_RUN_LARGE_PERF.
  • Measurements in the entry are from 59 real Codex rollouts (201 MB, largest 103 MB,
    release build): peak RSS 416 -> 66 MiB, parse 14.25 -> 2.16 s, byte-identical entry
    output before and after.

#184 moved Codex rollout parsing to a chunked FileHandle stream. The class
behind it lived only in that PR description, including which providers still
read whole files and why converting them is deferred. Record it in the defect
log with the two mechanics that make the streaming version work: splitting on
the 0x0A byte, and a per-chunk autorelease boundary. Note that the regression
guard is opt-in and does not run in CI.

Also add a step to the defect protocol: confirm a scenario is reachable before
adding or requesting a guard for it. Injection proves a guard works; it does
not prove the situation can occur.
@chattymin
chattymin merged commit 28560eb into main Aug 19, 2026
2 checks passed
@chattymin
chattymin deleted the docs/whole-file-read-class branch August 19, 2026 13:11
lukemonaghan pushed a commit to lukemonaghan/PokeTokenBar that referenced this pull request Aug 24, 2026
chattymin#192)

chattymin#184 moved Codex rollout parsing to a chunked FileHandle stream. The class
behind it lived only in that PR description, including which providers still
read whole files and why converting them is deferred. Record it in the defect
log with the two mechanics that make the streaming version work: splitting on
the 0x0A byte, and a per-chunk autorelease boundary. Note that the regression
guard is opt-in and does not run in CI.

Also add a step to the defect protocol: confirm a scenario is reachable before
adding or requesting a guard for it. Injection proves a guard works; it does
not prove the situation can occur.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant