Skip to content

shell args: bash has no formal parameter list -- both the ground-truth measurement and GitGalaxy's own counting were incomplete #1518

Description

@squid-protocol

Summary

args_exact_match for shell measured 0/3 (0%) in docs/self_scan/tree_sitter_accuracy_history.csv -- the worst score in the whole table, which reads as "GitGalaxy's shell args regex is badly broken." It wasn't -- both the measurement and the engine had a real, fixable gap, but neither was actually broken in the way the raw percentage suggested.

Root cause 1 (measurement): ground truth only checked the declaration

Bash functions (foo() { ... }) have NO formal parameter-list syntax at all, permanently, by grammar -- the () are always empty. tests/tools/tree_sitter_accuracy_audit.py's _get_param_count only checked the declaration node for a formal parameter field, correctly found nothing, and reported real=0 for every bash function regardless of true arity. GitGalaxy's own args regex instead reads the BODY for positional-parameter references ($1, $2, "$@"), which is the only way arguments are ever expressed in bash -- comparing that against a ground truth that only ever looked at the (permanently empty) declaration was an unfair, misleading comparison.

Root cause 2 (engine): single match only ever saw the first positional reference

Confirmed via brew/brew::symlink_target_directory, which references BOTH $1 and $2 in its body but measured got=1 -- GitGalaxy's args-counting takes one .search() per function and stops at the first match, so every positional-parameter reference after the first was silently dropped.

Fix (implemented, see PR)

  • tests/tools/tree_sitter_accuracy_audit.py: new _count_shell_real_positional_max, walking the function body (via tree-sitter's own real parse) for every simple_expansion/expansion node, returning the highest positional index referenced. $0 (script name, not an argument) is correctly excluded.
  • gitgalaxy/core/detector.py: new _count_shell_positional_max + a new opt-in _args_findall_max_groups routing flag, so shell's args-counting scans the WHOLE block (not just the first match) and takes the max index referenced.
  • gitgalaxy/standards/language_standards.py: shell's args regex wrapped in one outer capture group so the new routing flag can target it.

args_exact_match: 0/3 (0%) -> 3/3 (100%).

Why this isn't "GitGalaxy loses to tree-sitter here"

For this one specific signal (parameter count), in this one specific circumstance (no formal parameter-list syntax exists at all), a plain AST/tree-sitter-only tool has nothing to read at the declaration and can only ever report 0 -- correct, but useless for coupling/complexity scoring. GitGalaxy's body-aware regex is the more informative measurement here. Full writeup: docs/why_gitgalaxy_beats_ast_here.md.

This only affects extraction-pillar args-counting accuracy (epic #813) and the args_exact_match measurement (epic #1227/#1261), not risk scoring beyond the signal's own coupling number.

Activity

  1. added
    bugUnintended behavior or logic failure in the engine
    testingUnit, integration, and E2E pipeline verification
    core-engineModifications to the central physics and parsing engine
    threat: false-negativeAccuracy tuning: Engine missed a bypass or vulnerability
    on Aug 14, 2026
  2. added 2 commits that reference this issue on Aug 14, 2026
    cb59c23
    19d14b6
  3. added this to the v2.4.8 milestone on Aug 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugUnintended behavior or logic failure in the enginecore-engineModifications to the central physics and parsing enginetestingUnit, integration, and E2E pipeline verificationthreat: false-negativeAccuracy tuning: Engine missed a bypass or vulnerability

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions