From 47973a67955731bca584981d2cfb05df959e0a68 Mon Sep 17 00:00:00 2001 From: Joe Esquibel Date: Wed, 12 Aug 2026 13:37:43 -0400 Subject: [PATCH 1/3] fix(zig): wire zig into class_start named-entity extraction (#1295) Two fixes, plus a documented set of ground-truth-vs-regex scope gaps (same discipline as csharp/go's precedent in #1264): 1. Ground-truth gap: zig's error sets (`const Foo = error{...};`) get their own `ErrorSetDecl` node type, distinct from `ContainerDecl` -- a real class-like entity GitGalaxy's own class_start regex intentionally matches (its own comment: "struct, enum, union, error, opaque"), but NODE_MAPS["zig"] didn't include it. Added, dispatched through the same _get_zig_container_name resolver ContainerDecl already uses. This alone dropped extra_classes from 10 -> 1. 2. class_start regex gap: an optional `\(` and `\*` step-over added between `=` and the packed/extern modifier, confirmed against two real corpus idioms -- a parenthesized-wrapped anonymous struct (`const lenAsc = (struct { ... }).lenAsc;`) and a pointer-to-opaque handle type (`const HMONITOR = *opaque {};`, the standard idiom for an opaque OS handle). The remaining extra_classes=1 ("Borrowed" in bun/MimallocArena.zig) is a confirmed tree-sitter-zig grammar limitation: the file uses Zig's newer `#field` private-field syntax, which the grammar can't parse (has_error=True on that node), so error recovery swallows the container declaration -- the same shape as csharp's LanguageParser precedent in #1264. Hand-blessed tests/tree_sitter_accuracy_baseline_zig.json to the measured numbers (found_classes 0->630, real_classes 642->656, extra_classes 0->1) since --regenerate correctly refuses on a raw extra_classes regression and this one isn't a GitGalaxy bug. missing_classes (23->26, net of the 2 regex fixes minus 5 newly-exposed-by -the-ErrorSetDecl-fix compound error-union-chain cases) stays non-zero by design -- documented per-category in tests/extraction/how_to_extend_class_start_named_extraction.md rather than chased in this PR: compound error-set unions (`A || B || error{C}`), if/else conditional anonymous types, type-annotation-position anonymous types (`var x: enum {...} = ...`), and anonymous structs nested inside a generic call argument or array-element type (tree-sitter's ancestor walk finding them regardless of whether they're the direct declared value). zig added to detector.py's _CLASS_START_NAMED_EXTRACTION_LANGS allowlist. Golden master updated (both modes): comptime_string_map.zig's class count 6->7 (the parenthesized-struct fix) and win32.zig's 158->159 (the pointer-to-opaque fix) -- both intentional, all other diffs are downstream numeric fallout of those two signal changes. Verified: crucible_check.py (both modes) clean after regenerating the golden master, core_engine/extraction suite (6584 tests) passes, audit_check.py clean (ruff baseline regenerated, pure line-shifts only). Co-Authored-By: Claude Sonnet 5 --- gitgalaxy/core/detector.py | 1 + gitgalaxy/standards/language_standards.py | 11 ++- tests/golden_master_audit.json | 74 ++++++++++---------- tests/golden_master_zero_dep_audit.json | 74 ++++++++++---------- tests/ruff_audit_baseline.json | 10 +-- tests/tools/tree_sitter_accuracy_audit.py | 10 ++- tests/tree_sitter_accuracy_baseline_zig.json | 6 +- 7 files changed, 100 insertions(+), 86 deletions(-) diff --git a/gitgalaxy/core/detector.py b/gitgalaxy/core/detector.py index a1c89818b7..8b40d72f27 100644 --- a/gitgalaxy/core/detector.py +++ b/gitgalaxy/core/detector.py @@ -389,6 +389,7 @@ def get_mode(cls, lang_id: str) -> Optional[str]: "swift", "tcl", "typescript", + "zig", } ) diff --git a/gitgalaxy/standards/language_standards.py b/gitgalaxy/standards/language_standards.py index 68960c6bb5..d823babef9 100644 --- a/gitgalaxy/standards/language_standards.py +++ b/gitgalaxy/standards/language_standards.py @@ -64,7 +64,7 @@ | Swift | 98.9% | 100.0% | 100.0% | 100.0% | | Tcl | 98.6% | 98.6% | N/A | N/A | | Typescript | 92.7% | 89.8% | 100.0% | 100.0% | -| Zig | 73.9% | 100.0% | 0.0% | N/A | +| Zig | 73.9% | 100.0% | 96.0% | 99.8% | """ @@ -8604,8 +8604,15 @@ class PrismConfigSchema(TypedDict): re.M, ), # 5. class_start: Object / Entity Declarations. Defines structural entities (struct, enum, union, error, opaque). + # #1295: an optional `\(` and/or `\*` step-over added between `=` + # and the packed/extern modifier so two real corpus-confirmed + # idioms aren't missed: a parenthesized-wrapped anonymous struct + # (`const lenAsc = (struct { ... }).lenAsc;`, common for a + # struct-as-namespace immediately projecting one declaration out + # of itself) and a pointer-to-opaque handle type (`const HMONITOR + # = *opaque {};`, the standard idiom for an opaque OS handle). "class_start": re.compile( - r"^[ \t]*(?:pub[ \t\n]+)?const[ \t\n]+(@\"[^\"]+\"|[a-zA-Z_]\w*)[ \t\n]*=[ \t\n]*(?:packed[ \t\n]+|extern[ \t\n]+)?(?:struct|enum|union|error|opaque)(?=[ \t\n]*[{(])", + r"^[ \t]*(?:pub[ \t\n]+)?const[ \t\n]+(@\"[^\"]+\"|[a-zA-Z_]\w*)[ \t\n]*=[ \t\n]*\(?[ \t\n]*\*?[ \t\n]*(?:packed[ \t\n]+|extern[ \t\n]+)?(?:struct|enum|union|error|opaque)(?=[ \t\n]*[{(])", re.M, ), # --- PHASE 2: RISK & STRUCTURAL INTEGRITY --- diff --git a/tests/golden_master_audit.json b/tests/golden_master_audit.json index ed642f8689..08b302d42e 100644 --- a/tests/golden_master_audit.json +++ b/tests/golden_master_audit.json @@ -12,8 +12,8 @@ }, "Target Root Name": "data", "Absolute Project Path": "/home/joe/nyx_projects/language-crucible/data", - "Analysis ISO Timestamp": "2026-08-12T17:12:16.605840+00:00", - "Total Scan Duration": "35.22 seconds" + "Analysis ISO Timestamp": "2026-08-12T17:32:54.529721+00:00", + "Total Scan Duration": "37.44 seconds" }, "Source Control Footprint (Immutable Anchor)": { "Active Branch": "HEAD", @@ -282629,26 +282629,26 @@ }, "3. Architectural Profile": { "Repository Archetype": "file_cluster_11", - "Repository Drift (Z-Score)": 11.427, + "Repository Drift (Z-Score)": 11.434, "Repository Fingerprint": { - "file_cluster_0": 12.297, - "file_cluster_1": 12.737, - "file_cluster_2": 12.678, - "file_cluster_3": 12.13, - "file_cluster_4": 12.817, - "file_cluster_5": 13.06, - "file_cluster_6": 12.351, - "file_cluster_7": 12.287, - "file_cluster_8": 12.15, - "file_cluster_9": 12.557, - "file_cluster_10": 12.938, - "file_cluster_11": 11.427, - "file_cluster_12": 12.209, - "file_cluster_13": 12.237, - "file_cluster_14": 14.297, - "file_cluster_15": 12.507, - "file_cluster_16": 12.017, - "file_cluster_17": 12.559 + "file_cluster_0": 12.304, + "file_cluster_1": 12.743, + "file_cluster_2": 12.684, + "file_cluster_3": 12.137, + "file_cluster_4": 12.823, + "file_cluster_5": 13.066, + "file_cluster_6": 12.357, + "file_cluster_7": 12.293, + "file_cluster_8": 12.156, + "file_cluster_9": 12.563, + "file_cluster_10": 12.944, + "file_cluster_11": 11.434, + "file_cluster_12": 12.215, + "file_cluster_13": 12.244, + "file_cluster_14": 14.302, + "file_cluster_15": 12.513, + "file_cluster_16": 12.024, + "file_cluster_17": 12.566 }, "File Archetype": null, "File Drift (Z-Score)": 0.0, @@ -282917,7 +282917,7 @@ "Sequential Logic Declarations": 58, "Function Parameters": 23, "Function/Method Declarations": 23, - "Class/Entity Declarations": 6, + "Class/Entity Declarations": 7, "Defensive Programming Constructs": 26, "Type/Safety Bypasses": 5, "High-Risk Execution Commands": 0, @@ -287160,26 +287160,26 @@ }, "3. Architectural Profile": { "Repository Archetype": "file_cluster_11", - "Repository Drift (Z-Score)": 12.049, + "Repository Drift (Z-Score)": 12.05, "Repository Fingerprint": { - "file_cluster_0": 12.593, - "file_cluster_1": 12.833, + "file_cluster_0": 12.594, + "file_cluster_1": 12.834, "file_cluster_2": 12.874, - "file_cluster_3": 12.447, - "file_cluster_4": 13.151, + "file_cluster_3": 12.448, + "file_cluster_4": 13.152, "file_cluster_5": 13.219, - "file_cluster_6": 12.547, + "file_cluster_6": 12.548, "file_cluster_7": 12.39, - "file_cluster_8": 12.129, - "file_cluster_9": 12.793, + "file_cluster_8": 12.13, + "file_cluster_9": 12.794, "file_cluster_10": 13.267, - "file_cluster_11": 12.049, - "file_cluster_12": 12.815, + "file_cluster_11": 12.05, + "file_cluster_12": 12.816, "file_cluster_13": 12.278, - "file_cluster_14": 14.666, - "file_cluster_15": 12.92, - "file_cluster_16": 12.585, - "file_cluster_17": 13.025 + "file_cluster_14": 14.667, + "file_cluster_15": 12.921, + "file_cluster_16": 12.586, + "file_cluster_17": 13.026 }, "File Archetype": null, "File Drift (Z-Score)": 0.0, @@ -288668,7 +288668,7 @@ "Sequential Logic Declarations": 171, "Function Parameters": 403, "Function/Method Declarations": 181, - "Class/Entity Declarations": 158, + "Class/Entity Declarations": 159, "Defensive Programming Constructs": 4, "Type/Safety Bypasses": 154, "High-Risk Execution Commands": 1, diff --git a/tests/golden_master_zero_dep_audit.json b/tests/golden_master_zero_dep_audit.json index 7d07dc25f2..2f913163f3 100644 --- a/tests/golden_master_zero_dep_audit.json +++ b/tests/golden_master_zero_dep_audit.json @@ -12,8 +12,8 @@ }, "Target Root Name": "data", "Absolute Project Path": "/home/joe/nyx_projects/language-crucible/data", - "Analysis ISO Timestamp": "2026-08-12T17:13:08.322240+00:00", - "Total Scan Duration": "33.57 seconds" + "Analysis ISO Timestamp": "2026-08-12T17:33:48.311394+00:00", + "Total Scan Duration": "35.28 seconds" }, "Source Control Footprint (Immutable Anchor)": { "Active Branch": "HEAD", @@ -282629,26 +282629,26 @@ }, "3. Architectural Profile": { "Repository Archetype": "file_cluster_11", - "Repository Drift (Z-Score)": 11.427, + "Repository Drift (Z-Score)": 11.434, "Repository Fingerprint": { - "file_cluster_0": 12.297, - "file_cluster_1": 12.737, - "file_cluster_2": 12.678, - "file_cluster_3": 12.13, - "file_cluster_4": 12.817, - "file_cluster_5": 13.06, - "file_cluster_6": 12.351, - "file_cluster_7": 12.287, - "file_cluster_8": 12.15, - "file_cluster_9": 12.557, - "file_cluster_10": 12.938, - "file_cluster_11": 11.427, - "file_cluster_12": 12.209, - "file_cluster_13": 12.237, - "file_cluster_14": 14.297, - "file_cluster_15": 12.507, - "file_cluster_16": 12.017, - "file_cluster_17": 12.559 + "file_cluster_0": 12.304, + "file_cluster_1": 12.743, + "file_cluster_2": 12.684, + "file_cluster_3": 12.137, + "file_cluster_4": 12.823, + "file_cluster_5": 13.066, + "file_cluster_6": 12.357, + "file_cluster_7": 12.293, + "file_cluster_8": 12.156, + "file_cluster_9": 12.563, + "file_cluster_10": 12.944, + "file_cluster_11": 11.434, + "file_cluster_12": 12.215, + "file_cluster_13": 12.244, + "file_cluster_14": 14.302, + "file_cluster_15": 12.513, + "file_cluster_16": 12.024, + "file_cluster_17": 12.566 }, "File Archetype": null, "File Drift (Z-Score)": 0.0, @@ -282917,7 +282917,7 @@ "Sequential Logic Declarations": 58, "Function Parameters": 23, "Function/Method Declarations": 23, - "Class/Entity Declarations": 6, + "Class/Entity Declarations": 7, "Defensive Programming Constructs": 26, "Type/Safety Bypasses": 5, "High-Risk Execution Commands": 0, @@ -287160,26 +287160,26 @@ }, "3. Architectural Profile": { "Repository Archetype": "file_cluster_11", - "Repository Drift (Z-Score)": 12.049, + "Repository Drift (Z-Score)": 12.05, "Repository Fingerprint": { - "file_cluster_0": 12.593, - "file_cluster_1": 12.833, + "file_cluster_0": 12.594, + "file_cluster_1": 12.834, "file_cluster_2": 12.874, - "file_cluster_3": 12.447, - "file_cluster_4": 13.151, + "file_cluster_3": 12.448, + "file_cluster_4": 13.152, "file_cluster_5": 13.219, - "file_cluster_6": 12.547, + "file_cluster_6": 12.548, "file_cluster_7": 12.39, - "file_cluster_8": 12.129, - "file_cluster_9": 12.793, + "file_cluster_8": 12.13, + "file_cluster_9": 12.794, "file_cluster_10": 13.267, - "file_cluster_11": 12.049, - "file_cluster_12": 12.815, + "file_cluster_11": 12.05, + "file_cluster_12": 12.816, "file_cluster_13": 12.278, - "file_cluster_14": 14.666, - "file_cluster_15": 12.92, - "file_cluster_16": 12.585, - "file_cluster_17": 13.025 + "file_cluster_14": 14.667, + "file_cluster_15": 12.921, + "file_cluster_16": 12.586, + "file_cluster_17": 13.026 }, "File Archetype": null, "File Drift (Z-Score)": 0.0, @@ -288668,7 +288668,7 @@ "Sequential Logic Declarations": 171, "Function Parameters": 403, "Function/Method Declarations": 181, - "Class/Entity Declarations": 158, + "Class/Entity Declarations": 159, "Defensive Programming Constructs": 4, "Type/Safety Bypasses": 154, "High-Risk Execution Commands": 1, diff --git a/tests/ruff_audit_baseline.json b/tests/ruff_audit_baseline.json index 2be3518caa..5680345ca5 100644 --- a/tests/ruff_audit_baseline.json +++ b/tests/ruff_audit_baseline.json @@ -7,11 +7,11 @@ "gitgalaxy/core/aperture.py:446: SIM102": "Use a single `if` statement instead of nested `if` statements", "gitgalaxy/core/aperture.py:502: SIM102": "Use a single `if` statement instead of nested `if` statements", "gitgalaxy/core/aperture.py:514: SIM102": "Use a single `if` statement instead of nested `if` statements", - "gitgalaxy/core/detector.py:1892: SIM102": "Use a single `if` statement instead of nested `if` statements", - "gitgalaxy/core/detector.py:1899: SIM102": "Use a single `if` statement instead of nested `if` statements", - "gitgalaxy/core/detector.py:2341: SIM108": "Use ternary operator `line_end = len(safe_code) if next_nl == -1 else next_nl + 1` instead of `if`-`else`-block", - "gitgalaxy/core/detector.py:3229: C403": "Unnecessary list comprehension (rewrite as a set comprehension)", - "gitgalaxy/core/detector.py:797: SIM102": "Use a single `if` statement instead of nested `if` statements", + "gitgalaxy/core/detector.py:1893: SIM102": "Use a single `if` statement instead of nested `if` statements", + "gitgalaxy/core/detector.py:1900: SIM102": "Use a single `if` statement instead of nested `if` statements", + "gitgalaxy/core/detector.py:2342: SIM108": "Use ternary operator `line_end = len(safe_code) if next_nl == -1 else next_nl + 1` instead of `if`-`else`-block", + "gitgalaxy/core/detector.py:3230: C403": "Unnecessary list comprehension (rewrite as a set comprehension)", + "gitgalaxy/core/detector.py:798: SIM102": "Use a single `if` statement instead of nested `if` statements", "gitgalaxy/core/guidestar_lens.py:139: C401": "Unnecessary generator (rewrite as a set comprehension)", "gitgalaxy/galaxyscope.py:1041: SIM102": "Use a single `if` statement instead of nested `if` statements", "gitgalaxy/galaxyscope.py:1132: SIM118": "Use `key in dict` instead of `key in dict.keys()`", diff --git a/tests/tools/tree_sitter_accuracy_audit.py b/tests/tools/tree_sitter_accuracy_audit.py index 528b98db5c..f1eef0369b 100755 --- a/tests/tools/tree_sitter_accuracy_audit.py +++ b/tests/tools/tree_sitter_accuracy_audit.py @@ -406,7 +406,13 @@ # no name at all of its own -- see _get_zig_container_name's docstring for why the real # name lives on an ancestor VarDecl instead. "func_node_types": {"FnProto"}, - "class_node_types": {"ContainerDecl"}, + # #1295: zig's error sets (`const Foo = error{...};`) get their own `ErrorSetDecl` node + # type, distinct from `ContainerDecl` -- a real class-like entity GitGalaxy's own + # class_start regex intentionally matches (its comment says so: "Defines structural + # entities (struct, enum, union, error, opaque)"). Same shape as `_get_zig_container_name` + # (the real name lives on the enclosing VarDecl, not the ErrorSetDecl itself) so it's + # dispatched through the same resolver in `_get_node_name`. + "class_node_types": {"ContainerDecl", "ErrorSetDecl"}, }, "apex": { "ts_lang": "apex", @@ -729,7 +735,7 @@ def _get_node_name(node: Any) -> Optional[str]: if child.type == "IDENTIFIER": return child.text.decode("utf8") return None - if node.type == "ContainerDecl": + if node.type in ("ContainerDecl", "ErrorSetDecl"): return _get_zig_container_name(node) # #1295: go's type_declaration (struct/interface/type-alias) has no "name" field of its own -- diff --git a/tests/tree_sitter_accuracy_baseline_zig.json b/tests/tree_sitter_accuracy_baseline_zig.json index 52bc84400e..76c48a8571 100644 --- a/tests/tree_sitter_accuracy_baseline_zig.json +++ b/tests/tree_sitter_accuracy_baseline_zig.json @@ -2,11 +2,11 @@ "args_comparable": 1828, "args_exact_match": 1274, "corpus_path": "language-crucible/data/zig", - "extra_classes": 0, + "extra_classes": 1, "extra_functions": 0, "files_scanned": 32, - "found_classes": 0, + "found_classes": 630, "found_functions": 1828, - "real_classes": 642, + "real_classes": 656, "real_functions": 2473 } From f3bc7a101ff87f7c80690fc84310f65437a453d5 Mon Sep 17 00:00:00 2001 From: Joe Esquibel Date: Wed, 12 Aug 2026 13:38:54 -0400 Subject: [PATCH 2/3] docs: update class_start extraction checklist for zig (#1295) Move zig from "ground truth already works" to "done", record the ErrorSetDecl ground-truth fix, the paren/pointer-sigil regex widening, and the confirmed tree-sitter parse-error edge case. New recurring-cause class 7 documents the four scope-mismatch shapes found but not chased (nested-in-generic-call, nested-in-array-element-type, type-annotation position, and binary error-set-union chains) so a future pass has the categorization ready instead of re-deriving it. Co-Authored-By: Claude Sonnet 5 --- ..._to_extend_class_start_named_extraction.md | 30 +++++++++++++++---- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/tests/extraction/how_to_extend_class_start_named_extraction.md b/tests/extraction/how_to_extend_class_start_named_extraction.md index 6721cc0612..7a3b7f047b 100644 --- a/tests/extraction/how_to_extend_class_start_named_extraction.md +++ b/tests/extraction/how_to_extend_class_start_named_extraction.md @@ -21,7 +21,7 @@ _CLASS_START_NAMED_EXTRACTION_LANGS = frozenset({ **c, css, dart, go, haskell, html, kotlin, objective-c, perl, ruby, rust, swift, zig**. Extending the allowlist to them -- one at a time, with real verification -- is what epic #1295 tracks, and what this doc + `tests/tools/class_start_diff.py` exist to make cheap. **go, objective-c, swift, -and kotlin are done** (flipped into the allowlist, see the table below); 9 remain. +kotlin, and zig are done** (flipped into the allowlist, see the table below); 8 remain. Read `_resolve_class_start_match` in `gitgalaxy/core/detector.py` before touching anything -- it's the exact name-resolution algorithm the live pipeline uses (prefer capture group 1, fall @@ -43,9 +43,9 @@ during #1295's scaffolding pass -- do not re-derive this, just apply the fixes b |---|---| | **go** | Fixed (#1295 PR): `type_declaration` has no `name` field, but its `type_spec` child does. Flipped into `_CLASS_START_NAMED_EXTRACTION_LANGS`: `found_classes` 0→69, `extra_classes` 0→0. The 15 tree-sitter-only "missing" are all plain type aliases (`type WaitStatus uint32`) that go's own `class_start` intentionally excludes (requires `struct`/`interface` to follow) -- a scope mismatch, not a recall bug, documented in the baseline commit. | | **objective-c** | Fixed (#1295 PR): `class_interface`/`class_implementation`/`class_declaration` have no `name` field; first positional `identifier` child is the class name (second is the superclass for `class_interface` only). Flipped into the allowlist: `found_classes` 0→5, `extra_classes` 0→0, a perfect match. | -| **zig** | Ground truth already fixed (`ContainerDecl` resolves via `_get_zig_container_name`, walking up to the enclosing `VarDecl`) -- `real_classes` reads 642. **Not yet flipped into the allowlist**: `extra_classes` is 10, `missing_classes` is 23 -- a real per-file diff is needed before concluding whether `class_start` over/under-matches zig's `const X = struct {...}` idiom. Moved to the "ground truth already works" list below. | **swift** | Fixed (PR #1361): `class_start` had a capture group added (name only had an unparenthesized `[a-zA-Z_]\w*` before), widened to an optional dotted chain for nested-type extensions (`extension AFError.ParameterEncoderFailureReason`). Also hit the ground-truth measurement gap: tree-sitter-swift's protocols are a separate `protocol_declaration` node type, not `class_declaration` -- added to `NODE_MAPS["swift"]["class_node_types"]`. `real_classes` 37→38, `found_classes` 23→38, `extra_classes` stays 0. Flipped into the allowlist. | | **kotlin** | Fixed (PR #1365): three separate bugs stacked on this one. (1) `class_start` had no capture group -- added group 1 for the main class/interface/object/enum-class branch and group 2 for companion object's own optional name (Fortran/Lua-shaped alternation). (2) `fun interface Foo` (SAM declarations, Kotlin 1.4+) wasn't recognized -- `fun` was missing from the modifier list, a real recall gap confirmed against okhttp's `fun interface Factory`. (3) ground-truth gap: kotlin's `object` declarations (including `actual`/`expect` multiplatform variants) get their own `object_declaration` node type, not `class_declaration` -- added to `NODE_MAPS["kotlin"]["class_node_types"]` and `_get_node_name`. `real_classes` 4→7, `found_classes` 3→7, `extra_classes` stays 0. Golden master updated (okhttp/Call.kt's class count went 1→2). Flipped into the allowlist. | +| **zig** | Fixed (PR TBD): the biggest/messiest one so far (32 corpus files, `real_classes` in the hundreds), and the first to land with `extra_classes`/`missing_classes` still non-zero by design after investigation -- same discipline as csharp's enum precedent, not a shortcut. Ground truth gap: zig's error sets (`const Foo = error{...};`) get their own `ErrorSetDecl` node type, distinct from `ContainerDecl` -- a class-like entity GitGalaxy's own `class_start` regex intentionally matches (its comment literally says "struct, enum, union, error, opaque"). Added to `NODE_MAPS["zig"]` and dispatched through the same `_get_zig_container_name` resolver. Also widened `class_start` with an optional `\(`/`\*` step-over for two confirmed corpus idioms: a parenthesized-wrapped anonymous struct (`const lenAsc = (struct {...}).lenAsc;`) and a pointer-to-opaque handle type (`const HMONITOR = *opaque {};`). Net result: `extra_classes` 10→1, `found_classes` 0→630 (was 0 because the *legacy fallback* regex structurally can't match zig's `const X = struct` shape at all -- zig's own `class_start` always had a capture group, it just wasn't on the allowlist). The remaining `extra_classes=1` ("Borrowed" in bun/MimallocArena.zig) is a confirmed tree-sitter-zig grammar limitation (the file uses Zig's newer `#field` private-field syntax, which the grammar can't parse -- `has_error=True` on that node -- so error recovery swallows the declaration), the same shape as csharp's `LanguageParser` precedent in #1264. `--regenerate` correctly refuses on a raw `extra_classes` regression (0→1) even though `found_classes` improved massively -- baseline was hand-blessed to the measured numbers, same as csharp's precedent. `missing_classes` (23→26 net) stays non-zero **by design**, categorized in the "recurring cause classes" section below rather than chased in this PR -- see class 7. | ### Needs a judgment call, not a quick field-name fix @@ -64,9 +64,7 @@ each fix as its own small commit/PR (it changes a shared measurement file, not ### Languages where ground truth already works -- go straight to the regex workflow below **c** (real_classes=79), **rust** (242), **dart** (167), **ruby** (9), -**haskell** (1, small corpus), **zig** (642, but `extra_classes`=10/ -`missing_classes`=23 needs a real per-file diff before concluding what's over/under-matching). -(**swift, kotlin** done -- see the "Done" table above.) +**haskell** (1, small corpus). (**swift, kotlin, zig** done -- see the "Done" table above.) ## Per-language workflow (once ground truth is trustworthy) @@ -141,6 +139,28 @@ each fix as its own small commit/PR (it changes a shared measurement file, not regex is. New class discovered during #1295's scaffolding pass -- go/kotlin/objective-c/zig all hit this (now fixed, see the "Done"/"ground truth already works" tables above). **Always check this first** before touching a language's `class_start` regex. +7. **The ground-truth *walk* over-reaches beyond the regex's declaration anchor** -- a variant of + class 5, discovered on zig (PR TBD). The ground-truth node exists, resolves to a real name, and + isn't a scope-*category* mismatch (it genuinely is a struct/enum/error-set) -- but the ancestor + walk that recovers the name (`_get_zig_container_name`, bounded to `max_hops`) finds it + regardless of *how deeply nested* the anonymous type is inside the enclosing declaration's + initializer expression, while GitGalaxy's regex only anchors on the type keyword sitting + *immediately* after `=` (plus a narrow, explicitly-added set of step-overs). Four concrete + shapes seen on zig, none fixed in that PR (documented as accepted gaps, not chased): + - **Nested inside a generic call's argument**: `const Extra = List(struct { u32 });` -- the + anonymous struct is a type *argument* to `List(...)`, not the direct RHS value. + - **Nested inside an array-element type**: `const targets = [_]struct {...}{...};` -- same + shape, the struct describes the array's element type, not `targets` itself. + - **Type-annotation position, not initializer position**: `var op: enum {...} = .example;` -- + the anonymous type sits between `:` and `=`, a different grammatical slot GitGalaxy's regex + (which only ever looks *after* `=`) doesn't parse at all. + - **A binary chain of named error sets terminating in a literal**: `const X = A || B || + error{C};` -- tree-sitter resolves the *whole* union's declared name correctly; GitGalaxy's + regex would need to skip an unbounded-looking `ident || ident || ...` chain before the + `error` keyword, meaningfully more regex complexity/ReDoS-review than the other step-overs. + Each is a real, understood gap -- worth a dedicated follow-up decision (especially the + type-annotation-position one, which recurred 6x in zig's own corpus) rather than folding + into whichever PR happens to discover it. ## Parallelizing across languages -- NOT fully independent, unlike epic #1069's per-language files From 0d3a9143c0e8070eacf23f27ad5fde6eafc716b6 Mon Sep 17 00:00:00 2001 From: Joe Esquibel Date: Wed, 12 Aug 2026 13:39:49 -0400 Subject: [PATCH 3/3] docs: fill in PR #1367 reference for zig (#1295) Co-Authored-By: Claude Sonnet 5 --- .../extraction/how_to_extend_class_start_named_extraction.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/extraction/how_to_extend_class_start_named_extraction.md b/tests/extraction/how_to_extend_class_start_named_extraction.md index 7a3b7f047b..82949ed65c 100644 --- a/tests/extraction/how_to_extend_class_start_named_extraction.md +++ b/tests/extraction/how_to_extend_class_start_named_extraction.md @@ -45,7 +45,7 @@ during #1295's scaffolding pass -- do not re-derive this, just apply the fixes b | **objective-c** | Fixed (#1295 PR): `class_interface`/`class_implementation`/`class_declaration` have no `name` field; first positional `identifier` child is the class name (second is the superclass for `class_interface` only). Flipped into the allowlist: `found_classes` 0→5, `extra_classes` 0→0, a perfect match. | | **swift** | Fixed (PR #1361): `class_start` had a capture group added (name only had an unparenthesized `[a-zA-Z_]\w*` before), widened to an optional dotted chain for nested-type extensions (`extension AFError.ParameterEncoderFailureReason`). Also hit the ground-truth measurement gap: tree-sitter-swift's protocols are a separate `protocol_declaration` node type, not `class_declaration` -- added to `NODE_MAPS["swift"]["class_node_types"]`. `real_classes` 37→38, `found_classes` 23→38, `extra_classes` stays 0. Flipped into the allowlist. | | **kotlin** | Fixed (PR #1365): three separate bugs stacked on this one. (1) `class_start` had no capture group -- added group 1 for the main class/interface/object/enum-class branch and group 2 for companion object's own optional name (Fortran/Lua-shaped alternation). (2) `fun interface Foo` (SAM declarations, Kotlin 1.4+) wasn't recognized -- `fun` was missing from the modifier list, a real recall gap confirmed against okhttp's `fun interface Factory`. (3) ground-truth gap: kotlin's `object` declarations (including `actual`/`expect` multiplatform variants) get their own `object_declaration` node type, not `class_declaration` -- added to `NODE_MAPS["kotlin"]["class_node_types"]` and `_get_node_name`. `real_classes` 4→7, `found_classes` 3→7, `extra_classes` stays 0. Golden master updated (okhttp/Call.kt's class count went 1→2). Flipped into the allowlist. | -| **zig** | Fixed (PR TBD): the biggest/messiest one so far (32 corpus files, `real_classes` in the hundreds), and the first to land with `extra_classes`/`missing_classes` still non-zero by design after investigation -- same discipline as csharp's enum precedent, not a shortcut. Ground truth gap: zig's error sets (`const Foo = error{...};`) get their own `ErrorSetDecl` node type, distinct from `ContainerDecl` -- a class-like entity GitGalaxy's own `class_start` regex intentionally matches (its comment literally says "struct, enum, union, error, opaque"). Added to `NODE_MAPS["zig"]` and dispatched through the same `_get_zig_container_name` resolver. Also widened `class_start` with an optional `\(`/`\*` step-over for two confirmed corpus idioms: a parenthesized-wrapped anonymous struct (`const lenAsc = (struct {...}).lenAsc;`) and a pointer-to-opaque handle type (`const HMONITOR = *opaque {};`). Net result: `extra_classes` 10→1, `found_classes` 0→630 (was 0 because the *legacy fallback* regex structurally can't match zig's `const X = struct` shape at all -- zig's own `class_start` always had a capture group, it just wasn't on the allowlist). The remaining `extra_classes=1` ("Borrowed" in bun/MimallocArena.zig) is a confirmed tree-sitter-zig grammar limitation (the file uses Zig's newer `#field` private-field syntax, which the grammar can't parse -- `has_error=True` on that node -- so error recovery swallows the declaration), the same shape as csharp's `LanguageParser` precedent in #1264. `--regenerate` correctly refuses on a raw `extra_classes` regression (0→1) even though `found_classes` improved massively -- baseline was hand-blessed to the measured numbers, same as csharp's precedent. `missing_classes` (23→26 net) stays non-zero **by design**, categorized in the "recurring cause classes" section below rather than chased in this PR -- see class 7. | +| **zig** | Fixed (PR #1367): the biggest/messiest one so far (32 corpus files, `real_classes` in the hundreds), and the first to land with `extra_classes`/`missing_classes` still non-zero by design after investigation -- same discipline as csharp's enum precedent, not a shortcut. Ground truth gap: zig's error sets (`const Foo = error{...};`) get their own `ErrorSetDecl` node type, distinct from `ContainerDecl` -- a class-like entity GitGalaxy's own `class_start` regex intentionally matches (its comment literally says "struct, enum, union, error, opaque"). Added to `NODE_MAPS["zig"]` and dispatched through the same `_get_zig_container_name` resolver. Also widened `class_start` with an optional `\(`/`\*` step-over for two confirmed corpus idioms: a parenthesized-wrapped anonymous struct (`const lenAsc = (struct {...}).lenAsc;`) and a pointer-to-opaque handle type (`const HMONITOR = *opaque {};`). Net result: `extra_classes` 10→1, `found_classes` 0→630 (was 0 because the *legacy fallback* regex structurally can't match zig's `const X = struct` shape at all -- zig's own `class_start` always had a capture group, it just wasn't on the allowlist). The remaining `extra_classes=1` ("Borrowed" in bun/MimallocArena.zig) is a confirmed tree-sitter-zig grammar limitation (the file uses Zig's newer `#field` private-field syntax, which the grammar can't parse -- `has_error=True` on that node -- so error recovery swallows the declaration), the same shape as csharp's `LanguageParser` precedent in #1264. `--regenerate` correctly refuses on a raw `extra_classes` regression (0→1) even though `found_classes` improved massively -- baseline was hand-blessed to the measured numbers, same as csharp's precedent. `missing_classes` (23→26 net) stays non-zero **by design**, categorized in the "recurring cause classes" section below rather than chased in this PR -- see class 7. | ### Needs a judgment call, not a quick field-name fix @@ -140,7 +140,7 @@ each fix as its own small commit/PR (it changes a shared measurement file, not all hit this (now fixed, see the "Done"/"ground truth already works" tables above). **Always check this first** before touching a language's `class_start` regex. 7. **The ground-truth *walk* over-reaches beyond the regex's declaration anchor** -- a variant of - class 5, discovered on zig (PR TBD). The ground-truth node exists, resolves to a real name, and + class 5, discovered on zig (PR #1367). The ground-truth node exists, resolves to a real name, and isn't a scope-*category* mismatch (it genuinely is a struct/enum/error-set) -- but the ancestor walk that recovers the name (`_get_zig_container_name`, bounded to `max_hops`) finds it regardless of *how deeply nested* the anonymous type is inside the enclosing declaration's