Skip to content

docs(agent): two sweeps are not a drift, and #479 said they were - #480

Merged
cevheri merged 1 commit into
mainfrom
docs/two-sweeps-not-a-drift
Aug 24, 2026
Merged

cevheri merged 1 commit into
mainfrom
docs/two-sweeps-not-a-drift

Conversation

@cevheri

@cevheri cevheri commented Aug 24, 2026

Copy link
Copy Markdown
Member

docs(agent): two sweeps are not a drift, and #479 said they were

Correcting my own claim. #479 removed a duplicate model table from docs/AGENT.md and
wrote that every one of its ten rows "had drifted from the canonical figures", naming
gemma4:26b at 46 s/180 s "against a measured 36 s/92 s".

Both were measured. The owner's own sweep dashboard matches the removed table almost
row for row — gemma4:26b 46 s/3:00, qwen3.8:latest 1:02/5:47, qwen3:14b 42 s/2:31 —
so those figures were a real sweep, not a decayed copy of another one.

What separates the two sets is the CONDITION, which #465 already recorded and which
docs/llms/methodology.md states under "Sustained load is part of the measurement":
running all 300 back to back takes about three and a half hours, a machine under that
load is slower, and one model scored 1/5 on a surface at 87 to 93 seconds a run and 5/5
an hour later at 36. The cells that sweep lost were re-measured rested, and the rested
figures are the published ones.

So the removal stands and the reason given for it did not. Two tables of one measurement
is still the defect — neither said which sweep it was, and a reader had no way to tell —
but the fault was an unlabelled condition rather than a wrong number, and calling one set
"measured" implied the other was not.

The correction matters beyond tidiness: in a repository whose whole claim is that a
setting traces to a measurement, "these numbers drifted" and "these numbers are from a
loaded machine" send the next person to different places. The first says fix the copy;
the second says label the condition, which methodology.md had already done and the
tables had not.

The commit message on #479 cannot be edited; this is the file that can.

Correcting my own claim. #479 removed a duplicate model table from `docs/AGENT.md` and
wrote that every one of its ten rows "had drifted from the canonical figures", naming
`gemma4:26b` at 46 s/180 s "against a measured 36 s/92 s".

Both were measured. The owner's own sweep dashboard matches the removed table almost
row for row — `gemma4:26b` 46 s/3:00, `qwen3.8:latest` 1:02/5:47, `qwen3:14b` 42 s/2:31 —
so those figures were a real sweep, not a decayed copy of another one.

What separates the two sets is the CONDITION, which #465 already recorded and which
`docs/llms/methodology.md` states under "Sustained load is part of the measurement":
running all 300 back to back takes about three and a half hours, a machine under that
load is slower, and one model scored 1/5 on a surface at 87 to 93 seconds a run and 5/5
an hour later at 36. The cells that sweep lost were re-measured rested, and the rested
figures are the published ones.

So the removal stands and the reason given for it did not. Two tables of one measurement
is still the defect — neither said which sweep it was, and a reader had no way to tell —
but the fault was an unlabelled condition rather than a wrong number, and calling one set
"measured" implied the other was not.

The correction matters beyond tidiness: in a repository whose whole claim is that a
setting traces to a measurement, "these numbers drifted" and "these numbers are from a
loaded machine" send the next person to different places. The first says fix the copy;
the second says label the condition, which `methodology.md` had already done and the
tables had not.

The commit message on #479 cannot be edited; this is the file that can.
@sonarqubecloud

Copy link
Copy Markdown

@cevheri
cevheri merged commit 1c5fe03 into main Aug 24, 2026
19 checks passed
@cevheri
cevheri deleted the docs/two-sweeps-not-a-drift branch August 24, 2026 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant