From 5610287d3d504fb4e0c3740cf3c8f4e4cb40110f Mon Sep 17 00:00:00 2001 From: Joe Esquibel Date: Thu, 6 Aug 2026 20:00:36 -0400 Subject: [PATCH] docs: back the speed claim with the fitted rate model, not a single example Replaces "a handful of 20M+ LOC outliers take a few minutes instead" with the actual fitted two-regime model from gitgalaxy-raw-output's 599-repo batch: flat ~0.11s below ~4,258 LOC (fixed overhead dominates), then time(s) ~= 3.36e-05 * LOC^0.969 above it (R^2=0.88, near-linear rather than degrading on large inputs). Both the knee and the exponent are derived from the batch data (compute_rate_model() in that repo), not eyeballed. Embeds the LOC-vs-time chart in "Proof, Not Just Claims" item 3, pulled from gitgalaxy-raw-output's self-updating speed_charts/latest/ path so it never goes stale here -- no manual re-embedding needed on future releases. Logged in docs/readme_evidence_roadmap.md's Shipped table per the readme-maintenance process. Co-Authored-By: Claude Sonnet 5 --- README.md | 18 ++++++++++++++---- docs/readme_evidence_roadmap.md | 1 + 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 8d8be0b8d3..40a9aa8c5a 100644 --- a/README.md +++ b/README.md @@ -35,10 +35,11 @@ in, not the clean single-language input most static-analysis tooling assumes. has to build first. * **Fast enough to run on every commit.** Most repositories scan in well under a minute — [Kubernetes](https://github.com/squid-protocol/gitgalaxy-raw-output/blob/main/v2.4.6/kubernetes/kubernetes_galaxy_llm.md), - 1.39M lines across Go, YAML, JSON, Shell, and Proto, scans end to end in 50.83 seconds. A - handful of 20M+ LOC outliers take a few minutes instead — see the - [104-repo benchmark](https://squid-protocol.github.io/gitgalaxy/03-01-claim-1-search-strategies/) - for the full spread, not just the favorable case. + 1.39M lines across Go, YAML, JSON, Shell, and Proto, scans end to end in 50.83 seconds. Scan + time is fit as two regimes across a 599-repo batch — flat overhead below ~4,258 LOC, then + `time(s) ≈ 3.36e-05 × LOC^0.969` above it (R²=0.88, near-linear, not degrading on large + inputs) — see [Proof, Not Just Claims](#proof-not-just-claims) for the chart and derivation, + not just this rounded headline. * **CI-native output, not a standalone report.** Every scan produces a SARIF file (drops straight into GitHub/GitLab security dashboards), a CycloneDX SBOM (dependency compliance), and a 0–100 risk-exposure score per file, folder, and repo. See [Benchmarks](#benchmarks) for @@ -183,6 +184,15 @@ Every "structural signature" and "AST-free" claim above is backed by three thing 2. **A true golden diff against real, unmodified production code.** [`language-crucible`](https://github.com/squid-protocol/language-crucible) is a pinned, tagged snapshot of ~120 real subdirectories pulled from major open-source projects — Godot's C++, the Roslyn C# compiler, curl, Kubernetes, Apollo 11's AGC flight software, and more — deliberately left disconnected and uncompilable, the same hostile state real repos are in. Every pull request that touches the parsing engine re-scans that entire corpus and diffs the output, field by field, against a checked-in snapshot (`tests/golden_master_audit.json`); a diff means the output changed on real code, and it has to be explained before it's accepted — not a smoke test, an actual golden-master comparison. See [`tests/README.md`](tests/README.md#5-golden-master-differential-testing-the-language-crucible) for exactly how this is wired into CI, and [language-crucible's own README](https://github.com/squid-protocol/language-crucible) for why that corpus is built the way it is. 3. **[Unedited raw scan output at real-world scale](https://github.com/squid-protocol/gitgalaxy-raw-output).** Where the golden-master corpus above proves correctness on ~120 curated adversarial paradigms, this repo is the complementary evidence that the engine actually runs, unmodified, across hundreds of independently-chosen real repositories — every `_galaxy_audit.json`, `_galaxy_master.db`, and `_galaxy_llm.md` the scanner produced, kept versioned per engine release. The corpus manifest pinning exactly which repos and commits were scanned currently covers a 323-repo subset of the larger batch archived there — stated plainly in that repo's own README rather than implied to be complete. +That same raw-output batch is what the speed claim above is fit from — every repo plotted, not just the favorable Kubernetes example: + +

+ GitGalaxy scan time vs. LOC across hundreds of repositories, log-log, both axes +

+ +Always the newest scanner version — full derivation and methodology in +[gitgalaxy-raw-output's Speed Telemetry section](https://github.com/squid-protocol/gitgalaxy-raw-output#speed-telemetry). +
diff --git a/docs/readme_evidence_roadmap.md b/docs/readme_evidence_roadmap.md index 0ef4611f5b..b4dc0511c0 100644 --- a/docs/readme_evidence_roadmap.md +++ b/docs/readme_evidence_roadmap.md @@ -46,3 +46,4 @@ before its evidence exists (add a row here instead of writing it into README.md | Stale test-count fixed repo-wide: README.md and tests/README.md both cited 2,491/2,536 per-signature tests; actual collected count is 3,649 | 2026-08-06 | README.md, tests/README.md, `.claude/skills/readme-maintenance/SKILL.md` | | "What Pain Point Does This Solve?" section, addressing external feedback that a first-time visitor couldn't tell within a minute whether GitGalaxy competes with CodeQL/Semgrep/SonarQube or does something else — explicit "this is not X" disambiguation linking to the comparison table, plus a real cited example (Kubernetes, 1.39M LOC, 50.83s scan) instead of an abstract claim | 2026-08-06 | README.md, right after the proof strip; also retitled "Whole-Repository Intelligence with a Security Layer" to lead with the "one graph, three consumers" framing instead of security-first branding | | Restore the "0 dependencies" claim (stat line + badge) | 2026-08-06, [#1104](https://github.com/squid-protocol/gitgalaxy/issues/1104) — `PyYAML` moved from `pyproject.toml`'s `dependencies` into an optional `gitgalaxy[yaml]` extra; the 3 lazy-import call sites now raise/warn with an actionable install message instead of a silent no-op or bare `ImportError` | README.md stat line + badge | +| Fitted two-regime scan-speed model, replacing the "a handful of 20M+ LOC outliers take a few minutes" hand-wave: flat ~0.11s below ~4,258 LOC (fixed overhead), `time(s) ≈ 3.36e-05 × LOC^0.969` above it (R²=0.88, 599 repos) — both the knee and the exponent are derived from `gitgalaxy-raw-output`'s own batch data (`tools/generate_speed_telemetry.py`'s `compute_rate_model()`), not eyeballed. LOC-vs-time chart embedded from that repo's self-updating `speed_charts/latest/loc_vs_time.png`, so it never goes stale here | 2026-08-06 | README.md "What Pain Point Does This Solve?" (speed bullet) and "Proof, Not Just Claims" (item 3, plus the embedded chart) |