squid-protocol · Aug 11, 2026 · Aug 11, 2026
@@ -0,0 +1,70 @@
+name: Tree-sitter Accuracy Audit
+
+# Baseline-gated regression check (same shape as ruff/mypy/dead-key-audit.yml and
+# ast-accuracy-audit.yml) for tests/tools/tree_sitter_accuracy_audit.py -- catches a
+# structural-signature regex change that quietly drops recall/precision against the
+# tree-sitter ground truth, across every language with a committed baseline.
+#
+# Scoped to paths that can actually move these numbers, unlike ast-accuracy-audit.yml's
+# repo-wide trigger -- this check cost ~1 minute locally across all 23 languages (one
+# language-crucible clone + one galaxyscope scan per language), cheap enough to run on
+# every relevant PR, but there's no reason to pay even that on a PR that only touches
+# e.g. gitgalaxy/recorders/.
+
+on:
+  pull_request:
+    branches:
+      - main
+    paths:
+      - "gitgalaxy/core/detector.py"
+      - "gitgalaxy/core/prism.py"
+      - "gitgalaxy/standards/language_standards.py"
+      - "tests/tools/tree_sitter_accuracy_audit.py"
+      - "tests/tree_sitter_accuracy_baseline_*.json"
+      - ".github/workflows/tree-sitter-accuracy-audit.yml"
+
+permissions:
+  contents: read
+
+jobs:
+  tree-sitter-accuracy-audit:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout GitGalaxy PR
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          path: gitgalaxy
+          persist-credentials: false
+
+      - name: Setup Python
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.12"
+          cache: "pip"
+
+      - name: Install GitGalaxy + tree-sitter-language-pack
+        run: |
+          cd gitgalaxy
+          python -m pip install --upgrade pip
+          pip install -e .
+          pip install tree-sitter-language-pack
+
+      # Same pin golden-crucible.yml uses -- this tool reuses that corpus rather than
+      # a fresh clone (see tests/tools/tree_sitter_accuracy_audit.py's own CORPUS section).
+      - name: Fetch Language Crucible corpus (pinned v1.0)
+        run: git clone --branch v1.0 --depth 1 https://github.com/squid-protocol/language-crucible.git language-crucible
+
+      - name: Run baseline-gated regression check across every baselined language
+        run: |
+          cd gitgalaxy
+          python tests/tools/tree_sitter_accuracy_audit.py --all --ci
+
+      - name: Verify the summary table matches the committed baselines
+        run: |
+          cd gitgalaxy
+          python tests/tools/tree_sitter_accuracy_audit.py --summary-table
+          if ! git diff --exit-code -- gitgalaxy/standards/language_standards.py; then
+            echo "::error::gitgalaxy/standards/language_standards.py's summary table is stale."
+            echo "Run 'python tests/tools/tree_sitter_accuracy_audit.py --summary-table' and commit the result."
+            exit 1
+          fi
@@ -0,0 +1,105 @@
+name: Tree-sitter Accuracy History
+
+# Push-to-main companion to tree-sitter-accuracy-audit.yml's PR-time gate. That gate only
+# proves the *baselines* (tests/tree_sitter_accuracy_baseline_<lang>.json -- the numbers
+# regressions are measured against) didn't get worse; those stay human-reviewed and only
+# change via `tree_sitter_accuracy_audit.py --regenerate`, same discipline as the golden
+# master fixtures. This workflow never touches them.
+#
+# What it DOES do, once a relevant change actually lands on main: live-measure every
+# baselined language again and append one row each to
+# docs/self_scan/tree_sitter_accuracy_history.csv (an accumulating time series, unlike the
+# gitignored self-scan DB -- this is meant to be graphed later), and regenerate the
+# Markdown summary table embedded in language_standards.py's docstring so it can't drift
+# from the committed baselines between now and the next manual --summary-table run. Opens
+# an auto-merged PR for both, same mechanism gitgalaxy.yml uses for the LLM architecture
+# brief -- see that workflow's own comments for why a PR-then-merge instead of a direct
+# push (branch protection), and why the loop guard below is load-bearing.
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - "gitgalaxy/core/detector.py"
+      - "gitgalaxy/core/prism.py"
+      - "gitgalaxy/standards/language_standards.py"
+      - "tests/tools/tree_sitter_accuracy_audit.py"
+      - "tests/tree_sitter_accuracy_baseline_*.json"
+
+permissions:
+  contents: read
+
+jobs:
+  tree-sitter-accuracy-history:
+    # Guards against the same infinite-loop shape documented in gitgalaxy.yml's
+    # full-report job: this workflow's own bot commit touches language_standards.py and
+    # the baseline-adjacent history CSV, which are both in the trigger's `paths` filter,
+    # so without this it would re-trigger itself on every merge of its own PR.
+    if: |
+      !startsWith(github.event.head_commit.message, 'docs: auto-update tree-sitter accuracy data')
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+      pull-requests: write
+    steps:
+      - name: Checkout GitGalaxy
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          path: gitgalaxy
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: Setup Python
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.12"
+          cache: "pip"
+
+      - name: Install GitGalaxy + tree-sitter-language-pack
+        run: |
+          cd gitgalaxy
+          python -m pip install --upgrade pip
+          pip install -e .
+          pip install tree-sitter-language-pack
+
+      - name: Fetch Language Crucible corpus (pinned v1.0)
+        run: git clone --branch v1.0 --depth 1 https://github.com/squid-protocol/language-crucible.git language-crucible
+
+      - name: Append accuracy history + regenerate summary table
+        run: |
+          cd gitgalaxy
+          python tests/tools/tree_sitter_accuracy_audit.py --history
+          python tests/tools/tree_sitter_accuracy_audit.py --summary-table
+
+      - name: Open PR with updated history + summary table
+        id: tree-sitter-history-pr
+        uses: squid-protocol/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # forked from peter-evans/create-pull-request v8.1.1
+        with:
+          token: ${{ secrets.AUTOMATION_PAT }}
+          path: gitgalaxy
+          commit-message: "docs: auto-update tree-sitter accuracy data"
+          title: "docs: auto-update tree-sitter accuracy data"
+          body: |
+            Automated update from the latest tree-sitter accuracy measurement on `main`.
+
+            - Appends one row per language to `docs/self_scan/tree_sitter_accuracy_history.csv`.
+            - Regenerates the summary table in `gitgalaxy/standards/language_standards.py`.
+
+            Never touches `tests/tree_sitter_accuracy_baseline_*.json` -- those stay
+            human-reviewed via `tree_sitter_accuracy_audit.py --regenerate`.
+          branch: auto/tree-sitter-accuracy-history
+          delete-branch: true
+          add-paths: |
+            docs/self_scan/tree_sitter_accuracy_history.csv
+            gitgalaxy/standards/language_standards.py
+
+      # Same rationale as gitgalaxy.yml's "Merge the LLM Brief PR" step -- nothing else
+      # gates this PR, so without an explicit merge it just sits mergeable-but-unmerged.
+      - name: Merge the tree-sitter accuracy history PR
+        if: steps.tree-sitter-history-pr.outputs.pull-request-number
+        env:
+          GH_TOKEN: ${{ secrets.AUTOMATION_PAT }}
+        run: |
+          PR="${{ steps.tree-sitter-history-pr.outputs.pull-request-number }}"
+          gh pr merge --squash "$PR" --repo ${{ github.repository }} || \
+            gh pr merge --squash --auto "$PR" --repo ${{ github.repository }}
@@ -0,0 +1,26 @@
+timestamp_utc,commit_sha,language,files_scanned,real_functions,found_functions,extra_functions,real_classes,found_classes,extra_classes,args_comparable,args_exact_match,func_recall_pct,func_precision_pct,class_recall_pct,class_precision_pct
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,c,31,0,0,1687,79,45,7,0,0,,0.0,57.0,86.5
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,cpp,29,87,5,1147,140,138,29,5,0,5.7,0.4,98.6,82.6
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,csharp,6,558,553,397,19,0,0,553,116,99.1,58.2,0.0,
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,css,4,0,0,2,0,0,0,0,0,,0.0,,
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,dart,7,722,526,394,167,161,8,526,243,72.9,57.2,96.4,95.3
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,go,14,855,817,0,0,0,0,817,298,95.6,100.0,,
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,groovy,0,0,0,0,0,0,0,0,0,,,,
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,haskell,7,94,4,2,1,1,0,4,0,4.3,66.7,100.0,100.0
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,html,13,0,0,0,0,0,0,0,0,,,,
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,java,7,266,233,0,35,9,0,233,81,87.6,100.0,25.7,100.0
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,javascript,18,557,535,193,29,29,1,535,478,96.1,73.5,100.0,96.7
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,kotlin,5,0,0,10,0,0,3,0,0,,0.0,,0.0
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,lua,0,0,0,0,0,0,0,0,0,,,,
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,objective-c,5,0,0,113,0,0,0,0,0,,0.0,,
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,perl,22,1005,710,1,0,0,1,710,120,70.6,99.9,,0.0
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,php,35,1701,1701,1,28,27,1,1701,484,100.0,99.9,96.4,96.4
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,powershell,6,0,0,82,0,0,6,0,0,,0.0,,0.0
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,ruby,7,117,0,12,9,7,2,0,0,0.0,0.0,77.8,77.8
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,rust,43,1457,1135,32,242,145,10,1135,134,77.9,97.3,59.9,93.5
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,scala,7,499,205,0,17,8,0,205,45,41.1,100.0,47.1,100.0
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,shell,1,3,3,2,0,0,0,3,0,100.0,60.0,,
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,solidity,7,95,85,6,7,0,10,85,17,89.5,93.4,0.0,0.0
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,swift,6,88,60,3,37,9,0,60,10,68.2,95.2,24.3,100.0
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,typescript,23,1689,1566,177,285,274,537,1566,379,92.7,89.8,96.1,33.8
+2026-08-11T13:08:14+00:00,013b5c2540ded1e66d88501c14210d95ecc7f480,zig,32,0,0,1828,0,0,0,0,0,,0.0,,
@@ -22,11 +22,16 @@
 source code across the repository.
 
 ## Tree-sitter Accuracy Audit Summary
-This data is automatically generated by `tests/tools/tree_sitter_accuracy_audit.py`.
+This data is automatically generated by `tests/tools/tree_sitter_accuracy_audit.py --summary-table`
+from the committed `tests/tree_sitter_accuracy_baseline_<lang>.json` files -- do not hand-edit the
+table below, regenerate it instead (a stale table fails the tree-sitter-accuracy-audit CI check).
 The script analyzes the GitGalaxy regex engine's performance against the empirical AST
 ground truth provided by `tree_sitter_language_pack`, using the `language-crucible` corpus.
 Recall is calculated as (found / real). Precision is calculated as (found / (found + extra)).
+A language only appears once it has a committed baseline -- see docs/self_scan/tree_sitter_accuracy_history.csv
+for the same metrics tracked over time across pushes to main.
 
+<!-- TREE_SITTER_ACCURACY_TABLE:BEGIN -->
 | Language | Func Recall | Func Precision | Class Recall | Class Precision |
 | -------- | ----------- | -------------- | ------------ | --------------- |
 | C | N/A | 0.0% | 57.0% | 86.5% |
@@ -42,7 +47,7 @@
 | Javascript | 96.1% | 73.5% | 100.0% | 96.7% |
 | Kotlin | N/A | 0.0% | N/A | 0.0% |
 | Lua | N/A | N/A | N/A | N/A |
-| Objective-c | N/A | 0.0% | N/A | N/A |
+| Objective-C | N/A | 0.0% | N/A | N/A |
 | Perl | 70.6% | 99.9% | N/A | 0.0% |
 | Php | 100.0% | 99.9% | 96.4% | 96.4% |
 | Powershell | N/A | 0.0% | N/A | 0.0% |
@@ -54,6 +59,7 @@
 | Swift | 68.2% | 95.2% | 24.3% | 100.0% |
 | Typescript | 92.7% | 89.8% | 96.1% | 33.8% |
 | Zig | N/A | 0.0% | N/A | N/A |
+<!-- TREE_SITTER_ACCURACY_TABLE:END -->
 """