Skip to content

Persist a satellite-aware spatial ledger for post-hoc correlation (#348) - #350

Merged
squid-protocol merged 1 commit into
mainfrom
feature/348-persisted-ledger-post-hoc-correlation
Jul 24, 2026
Merged

squid-protocol merged 1 commit into
mainfrom
feature/348-persisted-ledger-post-hoc-correlation

Conversation

@squid-protocol

Copy link
Copy Markdown
Owner

Summary

Fixes #348 (phase 2 of #346/#102). This is the piece that actually unblocks #105 and #106.

  • correlate_against_ledger() (new, in spatial_correlation.py): joins file_data["threat_locations"] (line numbers, persisted for every rule) against file_data["functions"] (satellites, which already carry file-global start_line/end_line) at query time. No schema change to threat_locations was made -- sarif_recorder.py reads threat_locations[key][idx] expecting a plain int line number, so embedding a literal satellite id in each entry would have broken SARIF output. The join uses data that already exists on both sides.
  • security_lens.py's missing position data: scan_content() computed line positions for io/high_risk_execution/llm_hooks/db_hooks (now also hardcoded_secrets) internally, for its own variable-echo taint check, then discarded them -- only counts and snippets were ever returned. Now returns "positions" too, folded into the shared threat_locations ledger by galaxyscope.py's Phase 5.5 (additive, same pattern as the [BUG/CORE-ENGINE] security_lens results silently clobber detector.py's spatially-corroborated sec_tainted_injection count #344 counts merge). db_hooks position data -- needed for # [FEATURE/SAST] Implement Spatial Radar for Database Injection Funnels #105 -- exists in the persisted ledger for the first time.
  • The three amplifier-only correlate() pairs (RCE/IO taint corroboration, the concurrency half of the Race Condition Radar, the OOM Bomb cascading-flux check) migrated onto the scope-aware primitive from phase 1, via a new apply_amplifier_correlations() -- for consistency, not urgency (a wrong-scope amplifier costs one review item, not the false-negative risk phase 1 targeted), and cheap now that satellite ranges are already threaded through _function_slice().
  • "The Active Hemorrhage" (hardcoded secrets near a logging/print sink) could never run inside detector.py at all -- its target key, sec_hardcoded_secrets, is only ever populated by security_lens.py, which hadn't run yet at that point in the pipeline (confirmed dead in [BUG/CORE-ENGINE] security_lens results silently clobber detector.py's spatially-corroborated sec_tainted_injection count #344/[FEATURE/CORE-ENGINE] Extract a shared, scope-aware spatial correlation primitive and make threat_locations the persisted ledger #346). Removed the dead block from coding_analysis() and reimplemented it as a genuine post-hoc correlation in galaxyscope.py, against the now-persisted ledger.

Test plan

  • test_spatial_correlation.py: 4 new tests for correlate_against_ledger() (cross-function rejection, same-function acceptance, missing-key safety, corroboration-style read).
  • test_galaxyscope.py: test_worker_folds_security_lens_positions_into_threat_locations (verified failing pre-fix, passing post-fix) and test_worker_amplifies_active_hemorrhage_post_hoc (drives a real detector.py debug_prints hit against a mocked security_lens secret finding through the full worker path; verified failing pre-fix -- 1 != 51 -- passing post-fix).
  • test_detector.py: replaced the old in-detector Active Hemorrhage test (which tested a mechanism that no longer exists there) with one proving detector.py now reports only the raw, unamplified count for that key.
  • python -m pytest tests/core_engine/ tests/security_auditing/ -q -- 439 passed, 1 xfailed (pre-existing, unrelated).
  • python -m pytest tests/ -q -- full suite, 832 passed, 1 xfailed.

🤖 Generated with Claude Code

Phase 2 of #346. detector.py's six correlate() calls, and the "Active
Hemorrhage" block specifically, could only ever run against
coding_analysis()'s transient spatial_map -- data that's gone the
moment coding_analysis() returns. Nothing outside detector.py could
correlate against it, which blocks #105 and #106: both need to
correlate signals from a POST-HOC pipeline stage (dev_agent_firewall.py
and ai_appsec_sensor.py both run over fully-assembled parsed_files,
well after coding_analysis() has returned).

- Added correlate_against_ledger() to spatial_correlation.py: joins
  file_data["threat_locations"] (line numbers, persisted for every
  rule) against file_data["functions"] (satellites, which already
  carry file-global start_line/end_line) at query time. No schema
  change to threat_locations was needed or made -- sarif_recorder.py
  reads threat_locations[key][idx] expecting a plain int line number,
  so mutating its shape to embed a literal satellite id would have
  broken SARIF output. The join happens on read, using data that
  already exists on both sides.

- security_lens.py's scan_content() computed line positions for
  io/high_risk_execution/llm_hooks/db_hooks (and now hardcoded_secrets)
  internally, for its own variable-echo taint check, then discarded
  them -- only counts and snippets were ever returned. Now returns
  "positions" too, folded into the shared threat_locations ledger by
  galaxyscope.py's Phase 5.5 (additive, same as the #344 counts merge).
  db_hooks position data (needed for #105) exists in the persisted
  ledger for the first time.

- Migrated the three amplifier-only correlate() pairs (RCE/IO taint
  corroboration, the concurrency half of the Race Condition Radar, and
  the OOM Bomb cascading-flux check) onto the scope-aware primitive
  from phase 1, via a new apply_amplifier_correlations(), for
  consistency -- a wrong-scope amplifier only costs one review item,
  not the false-negative risk phase 1 targeted, so this was lower
  urgency but cheap now that the primitive and satellite ranges are
  already threaded through _function_slice().

- "The Active Hemorrhage" (hardcoded secrets near a logging/print sink)
  could never run inside detector.py at all: its target key,
  "sec_hardcoded_secrets", is only ever populated by security_lens.py,
  which hadn't run yet at that point in the pipeline (confirmed dead in
  #344/#346). Removed the dead block from coding_analysis() and
  reimplemented it as a genuine post-hoc correlation in galaxyscope.py,
  against the now-persisted ledger, via correlate_against_ledger().

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

@squid-protocol
squid-protocol merged commit ce6ade0 into main Jul 24, 2026
22 checks passed
@squid-protocol
squid-protocol deleted the feature/348-persisted-ledger-post-hoc-correlation branch July 24, 2026 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE/CORE-ENGINE] Persist a satellite-aware spatial ledger for post-hoc correlation (#346 phase 2)

1 participant