Skip to content

Fix BBOTTarget pickle support for engine subprocess#2983

Merged
liquidsec merged 1 commit into
3.0from
fix-target-pickle
Mar 23, 2026
Merged

Fix BBOTTarget pickle support for engine subprocess#2983
liquidsec merged 1 commit into
3.0from
fix-target-pickle

Conversation

@liquidsec

Copy link
Copy Markdown
Collaborator

Summary

  • BBOTTarget contains a RadixTarget (Rust/C extension) which can't be pickled. This breaks the HTTP engine subprocess spawn at runtime with cannot pickle 'builtins.PyRadixTarget' object.
  • Add __getstate__/__setstate__ to BaseTarget and ScanBlacklist so the radix tree is reconstructed from event_seeds after deserialization.
  • Tests didn't catch this because BBOT_TESTING=True runs the engine in-process (same event loop), bypassing the pickle path entirely.
  • Added test_target_pickle to cover pickle round-trips.

@github-actions

Copy link
Copy Markdown
Contributor

📊 Performance Benchmark Report

Comparing 3.0 (baseline) vs fix-target-pickle (current)

📈 Detailed Results (All Benchmarks)

📋 Complete results for all benchmarks - includes both significant and insignificant changes

🧪 Test Name 📏 Base 📏 Current 📈 Change 🎯 Status
Bloom Filter Dns Mutation Tracking Performance 4.25ms 4.22ms -0.7%
Bloom Filter Large Scale Dns Brute Force 17.56ms 17.32ms -1.4%
Large Closest Match Lookup 356.39ms 354.74ms -0.5%
Realistic Closest Match Workload 189.19ms 191.01ms +1.0%
Event Memory Medium Scan 1768 B/event 1766 B/event -0.1%
Event Memory Large Scan 1757 B/event 1757 B/event +0.0%
Event Validation Full Scan Startup Small Batch 408.85ms 399.79ms -2.2%
Event Validation Full Scan Startup Large Batch 593.70ms 571.52ms -3.7%
Make Event Autodetection Small 31.15ms 30.38ms -2.5%
Make Event Autodetection Large 313.24ms 309.92ms -1.1%
Make Event Explicit Types 13.81ms 13.59ms -1.6%
Excavate Single Thread Small 4.006s 3.887s -3.0%
Excavate Single Thread Large 9.824s 9.350s -4.8%
Excavate Parallel Tasks Small 4.237s 4.058s -4.2%
Excavate Parallel Tasks Large 7.387s 7.147s -3.3%
Is Ip Performance 3.19ms 3.23ms +1.4%
Make Ip Type Performance 11.69ms 11.50ms -1.6%
Mixed Ip Operations 4.53ms 4.58ms +1.2%
Typical Queue Shuffle 62.39µs 61.51µs -1.4%
Priority Queue Shuffle 705.41µs 689.09µs -2.3%

🎯 Performance Summary

No significant performance changes detected (all changes <10%)


🐍 Python Version 3.11.15

aconite33 added a commit that referenced this pull request Mar 23, 2026
Remove our __getstate__/__setstate__ from BaseTarget; the upstream
fix-target-pickle branch handles this more cleanly (explicit state,
direct acl_mode reading, ScanBlacklist override, and a test).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
aconite33 added a commit that referenced this pull request Mar 23, 2026
Remove our __getstate__/__setstate__ from BaseTarget; the upstream
fix-target-pickle branch handles this more cleanly (explicit state,
direct acl_mode reading, ScanBlacklist override, and a test).
@codecov

codecov Bot commented Mar 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91%. Comparing base (450946f) to head (2032b04).
⚠️ Report is 2 commits behind head on 3.0.

Additional details and impacted files
@@          Coverage Diff          @@
##             3.0   #2983   +/-   ##
=====================================
+ Coverage     91%     91%   +1%     
=====================================
  Files        436     436           
  Lines      36892   36918   +26     
=====================================
+ Hits       33516   33546   +30     
+ Misses      3376    3372    -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@liquidsec liquidsec merged commit de31418 into 3.0 Mar 23, 2026
13 of 14 checks passed
@liquidsec liquidsec mentioned this pull request Jun 9, 2026
@ausmaster ausmaster deleted the fix-target-pickle branch June 11, 2026 01:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants