Repository navigation
docs(agent): two sweeps are not a drift, and #479 said they were - #480
Merged
Merged
Conversation
Correcting my own claim. #479 removed a duplicate model table from `docs/AGENT.md` and wrote that every one of its ten rows "had drifted from the canonical figures", naming `gemma4:26b` at 46 s/180 s "against a measured 36 s/92 s". Both were measured. The owner's own sweep dashboard matches the removed table almost row for row — `gemma4:26b` 46 s/3:00, `qwen3.8:latest` 1:02/5:47, `qwen3:14b` 42 s/2:31 — so those figures were a real sweep, not a decayed copy of another one. What separates the two sets is the CONDITION, which #465 already recorded and which `docs/llms/methodology.md` states under "Sustained load is part of the measurement": running all 300 back to back takes about three and a half hours, a machine under that load is slower, and one model scored 1/5 on a surface at 87 to 93 seconds a run and 5/5 an hour later at 36. The cells that sweep lost were re-measured rested, and the rested figures are the published ones. So the removal stands and the reason given for it did not. Two tables of one measurement is still the defect — neither said which sweep it was, and a reader had no way to tell — but the fault was an unlabelled condition rather than a wrong number, and calling one set "measured" implied the other was not. The correction matters beyond tidiness: in a repository whose whole claim is that a setting traces to a measurement, "these numbers drifted" and "these numbers are from a loaded machine" send the next person to different places. The first says fix the copy; the second says label the condition, which `methodology.md` had already done and the tables had not. The commit message on #479 cannot be edited; this is the file that can.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



docs(agent): two sweeps are not a drift, and #479 said they were
Correcting my own claim. #479 removed a duplicate model table from
docs/AGENT.mdandwrote that every one of its ten rows "had drifted from the canonical figures", naming
gemma4:26bat 46 s/180 s "against a measured 36 s/92 s".Both were measured. The owner's own sweep dashboard matches the removed table almost
row for row —
gemma4:26b46 s/3:00,qwen3.8:latest1:02/5:47,qwen3:14b42 s/2:31 —so those figures were a real sweep, not a decayed copy of another one.
What separates the two sets is the CONDITION, which #465 already recorded and which
docs/llms/methodology.mdstates under "Sustained load is part of the measurement":running all 300 back to back takes about three and a half hours, a machine under that
load is slower, and one model scored 1/5 on a surface at 87 to 93 seconds a run and 5/5
an hour later at 36. The cells that sweep lost were re-measured rested, and the rested
figures are the published ones.
So the removal stands and the reason given for it did not. Two tables of one measurement
is still the defect — neither said which sweep it was, and a reader had no way to tell —
but the fault was an unlabelled condition rather than a wrong number, and calling one set
"measured" implied the other was not.
The correction matters beyond tidiness: in a repository whose whole claim is that a
setting traces to a measurement, "these numbers drifted" and "these numbers are from a
loaded machine" send the next person to different places. The first says fix the copy;
the second says label the condition, which
methodology.mdhad already done and thetables had not.
The commit message on #479 cannot be edited; this is the file that can.