Skip to content

feat: use literal shielding for precise AST-free calls_out_to extract… - #3258

Merged
squid-protocol merged 1 commit into
mainfrom
feat/ast-free-calls-out-extraction
Sep 21, 2026
Merged

squid-protocol merged 1 commit into
mainfrom
feat/ast-free-calls-out-extraction

Conversation

@squid-protocol

@squid-protocol squid-protocol commented Sep 21, 2026 •

Copy link
Copy Markdown
Owner

Summary

This PR fundamentally upgrades the Information Flow Graph (calls_out_to) extraction to utilize the AST-Free Literal Shield, allowing us to safely remove the hard 20-match truncation limit and eliminate string/comment false positives.

What was fixed/changed

  • Literal Shield Integration: Routed the raw function blocks through _apply_literal_shield before executing the calls_out_to regex. This sanitizes the block, preventing words inside strings or comments from being falsely extracted as function calls.
  • Removed Truncation Limit: Because the regex is no longer vulnerable to massive false-positive bloat from large string literals, we safely removed the arbitrary 20-match truncation limit. The engine now maps the complete call graph topology for every function.
  • Deduplication Optimization (Fixes function_data.calls_out_to is non-deterministic: list(set)[:20] keeps a different subset per PYTHONHASHSEED #3236): Wrapped the final calls_out_to array in dict.fromkeys() to deduplicate identical calls within the same function block. This significantly reduces the storage payload of scan.db while preserving the exact topology of unique external dependencies.
  • Architectural Shift: This marks a major milestone in proving the viability of our AST-free mapping architecture, demonstrating that ReDoS-proof regex combined with literal shielding can accurately extract deep structural metrics without the overhead of Tree-Sitter.

Core Engine Modification Checklist

  • Golden Master Verification: Ran crucible_check.py to verify diffs.
  • Tri-Comparison Audit: Regenerated baseline to reflect improved AST-free precision.
  • Tree-Sitter Accuracy: Ran audit to ensure zero true-positive regressions.

@github-actions

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

@squid-protocol
squid-protocol force-pushed the feat/ast-free-calls-out-extraction branch from 70b4d78 to f3807bd Compare September 21, 2026 17:55
@squid-protocol
squid-protocol merged commit e612261 into main Sep 21, 2026
31 checks passed
@squid-protocol
squid-protocol deleted the feat/ast-free-calls-out-extraction branch September 21, 2026 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

function_data.calls_out_to is non-deterministic: list(set)[:20] keeps a different subset per PYTHONHASHSEED

1 participant