src/conductor/fleet/history.py::_read_full_log documents itself as:
Reading the file unboundedly, streamed line-by-line via file iteration (never loaded into memory as a single blob) avoids that without paying for a whole-file read at once.
That is true of the raw bytes and false of the result. The function accumulates every parsed event and returns them:
events: list[dict[str, Any]] = []
...
events.append(obj)
...
return eventsPeak memory is therefore the whole log's parsed representation — typically 3-6x the file size, since a dict per line costs considerably more than the JSON text it came from. Avoiding the single read() moves where the bytes are held, not how many.
Why it matters
_build_entry, the only consumer, needs aggregate totals and a terminal status. It never needs the events list itself, so this could fold into the read loop at O(1) memory.- The History screen reads up to
keep_last logs (default 200), and build_history_entries is called on on_mount — so this runs for every retained log before the screen paints.
- It is inconsistent with
fleet/summary.py, which caps its full read at 8 MiB precisely to stop "a pathologically large log ... exhaust[ing] memory when a user opens the detail screen". Two modules away, the same concern is handled and documented; here it isn't.
Suggested direction
Fold the accumulation into the read loop and return the totals rather than the events. The one subtlety worth preserving is the saw_nonblank_line and not events check that raises _CorruptEventLogError: distinguishing "genuinely empty" from "non-empty but unparseable" is load-bearing for build_history_entries' skip logic, and needs an equivalent once the list is gone.
If the streaming shape is kept for other reasons, the docstring should say what it actually bounds.
Provenance
Raised in the review of #431 (R6) and deferred as non-blocking. Not a regression — the behaviour shipped with the Fleet Manager.
src/conductor/fleet/history.py::_read_full_logdocuments itself as:That is true of the raw bytes and false of the result. The function accumulates every parsed event and returns them:
Peak memory is therefore the whole log's parsed representation — typically 3-6x the file size, since a
dictper line costs considerably more than the JSON text it came from. Avoiding the singleread()moves where the bytes are held, not how many.Why it matters
_build_entry, the only consumer, needs aggregate totals and a terminal status. It never needs the events list itself, so this could fold into the read loop at O(1) memory.keep_lastlogs (default 200), andbuild_history_entriesis called onon_mount— so this runs for every retained log before the screen paints.fleet/summary.py, which caps its full read at 8 MiB precisely to stop "a pathologically large log ... exhaust[ing] memory when a user opens the detail screen". Two modules away, the same concern is handled and documented; here it isn't.Suggested direction
Fold the accumulation into the read loop and return the totals rather than the events. The one subtlety worth preserving is the
saw_nonblank_line and not eventscheck that raises_CorruptEventLogError: distinguishing "genuinely empty" from "non-empty but unparseable" is load-bearing forbuild_history_entries' skip logic, and needs an equivalent once the list is gone.If the streaming shape is kept for other reasons, the docstring should say what it actually bounds.
Provenance
Raised in the review of #431 (R6) and deferred as non-blocking. Not a regression — the behaviour shipped with the Fleet Manager.