Repository navigation
Epic: Refactor calls_out_to extraction to support non-C language paradigms #3264
Description
GitGalaxy's Information Flow Graph relies on the calls_out_to metric to map internal modularity (subroutines) and cross-module dependencies. Currently, detector.py uses a single generic bounded regex to extract these calls:
\b([a-zA-Z_]\w*)\s*\(
This regex fundamentally assumes that every programming language invokes functions using C-style syntax (name(arg)). A sweep of the keyword-rosetta baseline reveals that this assumption is severely breaking structural analysis for non-C paradigms.
The Impact
The single regex causes two distinct classes of failure:
-
Topology Blindness (Empty Graphs): Languages that use verbs, space-separation, or brackets for invocation never trigger the regex. Their
calls_out_toarrays remain empty, rendering their PageRank and dependency graphs useless.- Mainframes (
cobol,abap,assembly,hlasm): Rely on verbs likePERFORM,CALL,GO TO. - Shell Scripting (
shell,bash): Rely on space separation (command arg1). - Tcl & Obj-C: Rely on bracket dispatch (
[command arg1]).
- Mainframes (
-
Topology Pollution (Garbage Edges): Languages where control-flow constructs use parentheses are having their core keywords extracted as function targets.
- Fortran:
IF,ELSEIFare extracted as targets. - Perl:
my,elsifare extracted. - Scheme:
define,condare extracted. - COBOL: Intrinsic functions (
FUNCTION CURRENT-DATE()) and array subscripts (WS-TAB(I)) are falsely captured (Issue calls_out_to is meaningless for COBOL: generic name( pattern captures subscripts and intrinsics, never PERFORM/CALL targets #3202).
- Fortran:
Options to Solve
Option 1: The Regex Paradigm Switcher (Recommended)
Refactor the invocation_pattern in detector.py to route based on language paradigm, defining bespoke, high-precision regex patterns for each family:
C_FAMILY: Retain the current\b([a-zA-Z_]\w*)\s*\(but drastically enhance the language-specificignore_keywordslist to filter outIF,elsif, etc.MAINFRAME_VERBS:(?:PERFORM|CALL|GO\s+TO)\s+['"]?([A-Za-z0-9_-]+)LISP_FAMILY:\(\s*([a-zA-Z0-9_-]+)BRACKET_DISPATCH:\[\s*([a-zA-Z0-9_-]+)
Pros: Maintains the speed of AST-free bounded regex extraction. Fixes 90% of the gaps instantly.
Cons: Still struggles with Shell/Bash, where distinguishing a function call from a binary execution via regex is nearly impossible.
Option 2: AST-Assisted Extraction (Hybrid Mode)
Use Tree-Sitter (when available) to extract calls_out_to, falling back to regex only for unsupported languages.
Pros: Perfect precision for all languages.
Cons: Violates the strict AST-free engine requirement for the primary detector phase. Major performance penalty.
Option 3: Strict Opt-In Metric (The Lazy Route)
Mark calls_out_to as officially null (n/a) for non-C paradigms, actively deciding that GitGalaxy will not map dependency edges for Mainframes, Shell, or Lisp.
Pros: Easiest to implement. Instantly stops polluting the graph with garbage data.
Cons: A complete architectural cop-out that permanently blinds the product to the structure of enterprise mainframes.
Activity
squid-protocol commentedon Sep 21, 2026
Context & Reassurance on COBOL/Mainframe Paradigms:
An audit of
gitgalaxy/standards/language_standards/languages/cobol.pyconfirms that the actual structural regex keys for COBOL (class_start,func_start,import,args) are incredibly thorough and perfectly fitted to the mainframe paradigm (e.g., extractingPROGRAM-ID,COPY, andUSINGclauses).The language definitions themselves do not suffer from a systemic C-centric bias. The only reason
calls_out_tois failing so spectacularly across non-C paradigms is because it is a generic fallback hardcoded directly into thedetector.pyexecution loop, bypassing the paradigm-aware dictionary mechanism entirely.Implementing Option 1 (The Regex Paradigm Switcher) will simply bring
calls_out_toup to the same rigorous, paradigm-aware standard that the rest of the COBOL definitions already enjoy.feat: implement language-aware calls_out router and fix COBOL (Epic #…
test: update strict schemas + detector mock for the calls_out paradigm (
feat: batch map calls_out_to paradigms across all languages (Epic #3264…
feat: calls_out Phase 3 foundation — per-language ignore mechanism + …
squid-protocol commentedon Sep 22, 2026
Phase 3 complete (2026-09-22): per-language calls_out corrected across all 65 profiles
Option 1 (regex paradigm switcher) shipped via 6 engine PRs + 5 rosetta rebless PRs, gated end-to-end by a hand-derived truth oracle (keyword-rosetta#144:
docs/calls_out_truth.json+ checker, now PASS 51/51 on both repos' mains).Engine PRs: #3288 foundation (
_calls_out_ignoreunion mechanism + 3 shared constants) · #3289 demotions (11 data/markup langs → UNSUPPORTED) · #3294 CALL-verb + mainframe (fortran/pli/rexx/db2_sql/assembly + abap/hlasm/agc customs) · #3295 command-position + build tools (tcl/powershell/livecode + makefile/batch) · #3297 C-family ignore tuning (perl/ruby/lua/matlab/ada/m4) · #3298 functional/symbolic (scheme/haskell/objectivec/mlir/hlo) · #3296 helper-key convention fix.Rosetta: #144 oracle · #145–#149 reblesses (110+ cells re-baselined; rebless.py gained per-function diffs +
--add-keys functions:<metric>).Real-repo proof (zopeneditor-sample): COBOL paragraph PERFORM-chains (36 edges) and PL/I
PSAM1 → PRTHDG1/PSAM2/TRANTOT— graphs that were empty or garbage before.Gap register (all ledgered in the truth oracle):
name!(macros, abap=>methods, hlasm system macros, powershell$obj.method(), unary objc selectors, leading-::tcl, scheme operator applications🤖 Generated with Claude Code