Repository navigation
Persist a satellite-aware spatial ledger for post-hoc correlation (#348) - #350
Merged
squid-protocol merged 1 commit intoJul 24, 2026
Merged
Conversation
Phase 2 of #346. detector.py's six correlate() calls, and the "Active Hemorrhage" block specifically, could only ever run against coding_analysis()'s transient spatial_map -- data that's gone the moment coding_analysis() returns. Nothing outside detector.py could correlate against it, which blocks #105 and #106: both need to correlate signals from a POST-HOC pipeline stage (dev_agent_firewall.py and ai_appsec_sensor.py both run over fully-assembled parsed_files, well after coding_analysis() has returned). - Added correlate_against_ledger() to spatial_correlation.py: joins file_data["threat_locations"] (line numbers, persisted for every rule) against file_data["functions"] (satellites, which already carry file-global start_line/end_line) at query time. No schema change to threat_locations was needed or made -- sarif_recorder.py reads threat_locations[key][idx] expecting a plain int line number, so mutating its shape to embed a literal satellite id would have broken SARIF output. The join happens on read, using data that already exists on both sides. - security_lens.py's scan_content() computed line positions for io/high_risk_execution/llm_hooks/db_hooks (and now hardcoded_secrets) internally, for its own variable-echo taint check, then discarded them -- only counts and snippets were ever returned. Now returns "positions" too, folded into the shared threat_locations ledger by galaxyscope.py's Phase 5.5 (additive, same as the #344 counts merge). db_hooks position data (needed for #105) exists in the persisted ledger for the first time. - Migrated the three amplifier-only correlate() pairs (RCE/IO taint corroboration, the concurrency half of the Race Condition Radar, and the OOM Bomb cascading-flux check) onto the scope-aware primitive from phase 1, via a new apply_amplifier_correlations(), for consistency -- a wrong-scope amplifier only costs one review item, not the false-negative risk phase 1 targeted, so this was lower urgency but cheap now that the primitive and satellite ranges are already threaded through _function_slice(). - "The Active Hemorrhage" (hardcoded secrets near a logging/print sink) could never run inside detector.py at all: its target key, "sec_hardcoded_secrets", is only ever populated by security_lens.py, which hadn't run yet at that point in the pipeline (confirmed dead in #344/#346). Removed the dead block from coding_analysis() and reimplemented it as a genuine post-hoc correlation in galaxyscope.py, against the now-persisted ledger, via correlate_against_ledger(). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
5 tasks
Contributor
squid-protocol
deleted the
feature/348-persisted-ledger-post-hoc-correlation
branch
July 24, 2026 03:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #348 (phase 2 of #346/#102). This is the piece that actually unblocks #105 and #106.
correlate_against_ledger()(new, inspatial_correlation.py): joinsfile_data["threat_locations"](line numbers, persisted for every rule) againstfile_data["functions"](satellites, which already carry file-globalstart_line/end_line) at query time. No schema change tothreat_locationswas made --sarif_recorder.pyreadsthreat_locations[key][idx]expecting a plain int line number, so embedding a literal satellite id in each entry would have broken SARIF output. The join uses data that already exists on both sides.security_lens.py's missing position data:scan_content()computed line positions forio/high_risk_execution/llm_hooks/db_hooks(now alsohardcoded_secrets) internally, for its own variable-echo taint check, then discarded them -- only counts and snippets were ever returned. Now returns"positions"too, folded into the sharedthreat_locationsledger bygalaxyscope.py's Phase 5.5 (additive, same pattern as the [BUG/CORE-ENGINE] security_lens results silently clobber detector.py's spatially-corroborated sec_tainted_injection count #344 counts merge).db_hooksposition data -- needed for # [FEATURE/SAST] Implement Spatial Radar for Database Injection Funnels #105 -- exists in the persisted ledger for the first time.correlate()pairs (RCE/IO taint corroboration, the concurrency half of the Race Condition Radar, the OOM Bomb cascading-flux check) migrated onto the scope-aware primitive from phase 1, via a newapply_amplifier_correlations()-- for consistency, not urgency (a wrong-scope amplifier costs one review item, not the false-negative risk phase 1 targeted), and cheap now that satellite ranges are already threaded through_function_slice().detector.pyat all -- its target key,sec_hardcoded_secrets, is only ever populated bysecurity_lens.py, which hadn't run yet at that point in the pipeline (confirmed dead in [BUG/CORE-ENGINE] security_lens results silently clobber detector.py's spatially-corroborated sec_tainted_injection count #344/[FEATURE/CORE-ENGINE] Extract a shared, scope-aware spatial correlation primitive and make threat_locations the persisted ledger #346). Removed the dead block fromcoding_analysis()and reimplemented it as a genuine post-hoc correlation ingalaxyscope.py, against the now-persisted ledger.Test plan
test_spatial_correlation.py: 4 new tests forcorrelate_against_ledger()(cross-function rejection, same-function acceptance, missing-key safety, corroboration-style read).test_galaxyscope.py:test_worker_folds_security_lens_positions_into_threat_locations(verified failing pre-fix, passing post-fix) andtest_worker_amplifies_active_hemorrhage_post_hoc(drives a real detector.pydebug_printshit against a mocked security_lens secret finding through the full worker path; verified failing pre-fix --1 != 51-- passing post-fix).test_detector.py: replaced the old in-detector Active Hemorrhage test (which tested a mechanism that no longer exists there) with one proving detector.py now reports only the raw, unamplified count for that key.python -m pytest tests/core_engine/ tests/security_auditing/ -q-- 439 passed, 1 xfailed (pre-existing, unrelated).python -m pytest tests/ -q-- full suite, 832 passed, 1 xfailed.🤖 Generated with Claude Code