Repository navigation
tree_sitter_accuracy_audit.py reports real_functions=0 for css/kotlin/powershell/zig despite confirmed function-shaped nodes in real corpus files #1313
Description
What
Same measurement-tool-ground-truth class as #1265 (already fixed for C), found while
sanity-checking the full docs/language_standards.py summary table after #1266/#1307 landed.
Four committed baselines report real_functions: 0 (and css/zig also real_classes: 0) across
their entire corpus, alongside nonzero extra_functions -- i.e. the tool is confidently saying
"GitGalaxy found N phantom functions and there are zero real ones to compare against," which
given each language obviously has real functions in real code, points at the MEASUREMENT tool's
own tree-sitter query being broken for these four, not a GitGalaxy engine defect.
| Language | real_functions | found_functions | extra_functions |
|---|---|---|---|
| css | 0 | 0 | 2 |
| kotlin | 0 | 0 | 10 |
| powershell | 0 | 0 | 82 |
| zig | 0 | 0 | 1828 |
Evidence (root cause differs per language -- checked each directly against
tree_sitter_language_pack)
- css:
NODE_MAPS["css"]["func_node_types"] = {"at_rule"}in
tests/tools/tree_sitter_accuracy_audit.py-- but parsing a real@media (...) { ... }block
with this pack's css grammar produces a node of typemedia_statement, notat_rule.at_rule
does not appear anywhere in the parse tree at all. The configured node type is simply wrong for
this grammar version. - kotlin:
function_declaration(the configured type) DOES appear in the parse tree for 3 of
5 reallanguage-crucible/data/kotlinfiles (confirmed by walking the tree directly) -- so this
is NOT a wrong-node-type problem like css. The bug is somewhere in the tool's counting/
extraction logic for kotlin specifically (possibly a field-name lookup failing per [TIER 2] Verify tree-sitter accuracy measurement's own ground truth for C before treating found_functions=0 as a GitGalaxy defect #1265's exact
precedent for C, or an exception being silently swallowed per-file). - powershell: same shape as kotlin --
function_statement(the configured type) is present in
all 3 real corpus files, so the zero is a tool-logic bug, not a wrong node-type name. - zig:
FnProto(the configured type) IS present in a parse of a trivial real zig function --
so, like kotlin/powershell, the corpus-wide zero (across 32 files!) points at a tool-logic bug,
not a missing/wrong node type.
Suggested next step
Fix css's func_node_types to the correct node type (media_statement, and verify against a
broader real-corpus sample rather than just this one synthetic case -- css's own at-rules can
take other forms too, e.g. @keyframes/@font-face, worth checking each maps to the same or a
different node type). For kotlin/powershell/zig, instrument or step through measure()'s
per-file node-walking/counting logic against one real corpus file each to find why a confirmed-
present node type still counts as zero -- likely a #1265-shaped field-access assumption that
doesn't hold for these three grammars specifically.
Until fixed, none of these four languages' func_recall/func_precision numbers in the
language_standards.py summary table or tree_sitter_accuracy_history.csv can be trusted as a
real signal about GitGalaxy's own engine accuracy for them.
Related: #1261 (parent epic), #1265 (identical bug class, already fixed for C).
Activity
fix(tests): fix tree_sitter_accuracy_audit.py's zero real_functions/c…
docs: update #1295 checklist doc for go/objective-c completion + kotl…
fix(core): extend class_start named extraction to go and objective-c (#…
Validate css function/existence tri-comparison ledger shape (#2067)
Validate css function/existence tri-comparison ledger shape (#2069)
css tri-comparison: func precision + recall to 100% (audit-reader fix…
fix(ci): the census test imports the detonation harness via the langu…
fix(census): the undefined family resolved — css measured in, four jo…