You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Part of #3265. Depends on #3327 (contract) and #3328 (resolver). Required before any function-graph number is used in a README claim, a badge, or a headline score.
Two levels, scored separately:
Level 1: callee-name extraction. Precision/recall on (function, callee name) pairs per language, on the existing language-crucible corpus. Comparator: tree-sitter tagging queries (tags.scm@reference.call captures), which work at the same unresolved-name level. tree_sitter_language_pack is already in the audit venv. Reuse the tree-sitter accuracy audit's history CSV, chart and ledger pattern. ctags is not useful here: it indexes definitions, with little support for call references. Bonus: score the commit before #3267 against current main to get the missing before/after for Epic #3264.
Level 2: resolution correctness (the claim #3328 makes). On 3-4 flagship languages, compare resolved fcall edges with type-aware call graphs: PyCG or pyan3 (already in requirements.txt) for Python, and Sourcegraph SCIP indexers (scip-java, scip-typescript, scip-go) for the others. Report the wrong-target rate from name collisions. Mainframe: extend the hand-verified calls in tests/cobol_mainframe/answer_key/ to PL/I and JCL.
Control plants: add known-answer cases to keyword-rosetta:
calls inside strings and comments (must not count);
if ( / while ( look-alikes;
chained methods, constructors, functions passed by name.
Disagreements are verified against real source before they count (CLAUDE.md "comparative-correctness" rule).
Done when: a Level-1 CI audit is baselined for the corpus, and Level-2 numbers are published for at least Python plus one more language.
Activity
calls_out_tointo a First-Class Function Dependency Graph #3265Engine: report call-resolution rates per language and repo (#3331)
Engine: report call-resolution rates per language and repo (#3331) (#…
CI: gate call-graph Level-1 accuracy against its baseline (#3332)
Benchmark call-graph accuracy (#3332) + two resolver fixes it found (#…