Skip to content

tree_sitter_accuracy_audit.py: csharp ground truth misses a real method due to a tree-sitter parse error in a large corpus file #1427

Closed

Description

@squid-protocol

Summary

Found while independently verifying #1418 (csharp func_start Invocation Shield fix). After that
fix, tree_sitter_accuracy_audit.py --lang csharp's "Sample extra" output still lists
AccumulateExplicitInterfaceName from roslyn/LanguageParser.cs even though the fix reduces
GitGalaxy's own matches for that name from 4 down to exactly 1 (the real definition) -- confirmed
three independent ways: raw regex .finditer(), an isolated single-file galaxyscope scan, and a
direct SQL query against the corpus DB.

Root cause: tree_sitter_language_pack's C# grammar hits a parse error somewhere in this
14,680-line file (tree.root_node.has_error == True), and the resulting ground-truth parse tree
has no method_declaration node covering AccumulateExplicitInterfaceName at all. Since the
audit tool's "extra" classification is a straightforward set-difference against that ground-truth
set, ANY GitGalaxy match for this name -- including the single, correct one -- gets misclassified
as "extra" regardless of correctness.

This is the same class of issue as #1311/#1319 (audit-tool's own ground-truth gap, not a
GitGalaxy regex defect) -- filing per that established precedent rather than re-chasing it as a
GitGalaxy bug.

Suggested next step

Narrow down where in roslyn/LanguageParser.cs the C# grammar's parse error occurs (bisect the
file, or inspect tree.root_node.has_error/error node positions directly) to confirm the scope
of what else in this file's ground truth might be similarly undercounted -- this could mean
csharp's real extra_functions/args_exact_match numbers for this specific file are
systematically unreliable beyond just this one name, not just off by one entry.

Scope note

This is a measurement-tool defect in tests/tools/tree_sitter_accuracy_audit.py, not a
GitGalaxy structural-signature defect -- no language_standards.py/detector.py change implied.

Activity

  1. added
    bugUnintended behavior or logic failure in the engine
    testingUnit, integration, and E2E pipeline verification
    priority: lowUI tweaks, documentation, and minor optimizations
    on Aug 13, 2026
  2. added this to the v2.4.8 milestone on Aug 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugUnintended behavior or logic failure in the enginepriority: lowUI tweaks, documentation, and minor optimizationstestingUnit, integration, and E2E pipeline verification

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions