Skip to content

Dev -> Stable 3.0#3079

Merged
liquidsec merged 1407 commits into
stablefrom
dev
Jul 8, 2026
Merged

Dev -> Stable 3.0#3079
liquidsec merged 1407 commits into
stablefrom
dev

Conversation

@TheTechromancer

@TheTechromancer TheTechromancer commented May 4, 2026

Copy link
Copy Markdown
Contributor

Architecture / Core

Vulnerability / Finding Rework

Blasthttp

DNS

Performance

Lightfuzz

New Modules

Module Fixes / Removals

Config / Presets / Targets

UX / Output

Bug Fixes

Security

Documentation

Misc

Merge Conflict Resolution

Comment thread bbot/test/test_step_1/test_presets.py Dismissed
Comment thread bbot/test/test_step_1/test_presets.py Dismissed
Comment thread bbot/test/test_step_1/test_presets.py Dismissed
Comment thread bbot/test/test_step_1/test_presets.py Dismissed
Comment thread bbot/test/test_step_1/test_presets.py Dismissed
Comment thread bbot/test/test_step_1/test_presets.py Dismissed
Comment thread bbot/test/test_step_1/test_presets.py Fixed
Comment thread bbot/test/test_step_1/test_target.py Dismissed
Comment thread bbot/test/test_step_1/test_target.py Dismissed
Comment thread bbot/test/test_step_1/test_target.py Dismissed
@github-actions

github-actions Bot commented May 4, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@ChrisJr404

Copy link
Copy Markdown
Contributor

I have read the CLA Document and I hereby sign the CLA

@github-actions

github-actions Bot commented May 4, 2026

Copy link
Copy Markdown
Contributor

🚀 Performance Benchmark Report

ℹ️ No baseline benchmark data available

Showing current results for dev only.

📊 Current Results (dev) - Click to expand

Results

Test Name Mean Time Ops/sec Min Max
Bloom Filter Dns Mutation Tracking Performance 4.33ms 230.7 ops/sec 4.30ms 4.42ms
Bloom Filter Large Scale Dns Brute Force 18.42ms 54.3 ops/sec 18.10ms 18.91ms
Large Closest Match Lookup 369.95ms 2.7 ops/sec 366.30ms 372.42ms
Realistic Closest Match Workload 194.87ms 5.1 ops/sec 193.68ms 196.20ms
Event Memory Medium Scan 1.908s 0.5 ops/sec 1.792s 2.078s
Event Memory Large Scan 9.545s 0.1 ops/sec 9.422s 9.635s
Event Validation Full Scan Startup Small Batch 408.10ms 2.5 ops/sec 404.91ms 411.98ms
Event Validation Full Scan Startup Large Batch 545.19ms 1.8 ops/sec 543.27ms 546.87ms
Make Event Autodetection Small 25.91ms 38.6 ops/sec 25.79ms 26.05ms
Make Event Autodetection Large 264.34ms 3.8 ops/sec 263.34ms 265.55ms
Make Event Explicit Types 11.65ms 85.8 ops/sec 11.61ms 11.68ms
Excavate Single Thread Small 4.392s 0.2 ops/sec 4.222s 4.629s
Excavate Single Thread Large 10.222s 0.1 ops/sec 10.082s 10.410s
Excavate Parallel Tasks Small 4.408s 0.2 ops/sec 4.345s 4.611s
Excavate Parallel Tasks Large 6.618s 0.2 ops/sec 6.485s 6.815s
Intercept Throughput Small 887.14ms 1.1 ops/sec 876.57ms 907.43ms
Intercept Throughput Medium 872.58ms 1.1 ops/sec 867.18ms 879.64ms
Dns Throughput Quiet 2.758s 0.4 ops/sec 2.744s 2.769s
Dns Throughput Loaded 2.021s 0.5 ops/sec 1.911s 2.130s
Dns Throughput Inherited 2.401s 0.4 ops/sec 2.354s 2.433s
Is Ip Performance 2.30ms 434.3 ops/sec 2.19ms 3.42ms
Make Ip Type Performance 246.37µs 4.1K ops/sec 222.65µs 814.09µs
Mixed Ip Operations 2.40ms 417.2 ops/sec 2.28ms 2.90ms
Memory Use Web Crawl 712ns 1404.5K ops/sec 712ns 712ns
Memory Use Subdomain Enum 642ns 1557.6K ops/sec 642ns 642ns
Memory Use Deep Chain 701ns 1426.5K ops/sec 701ns 701ns
Memory Use Parallel Chains 822ns 1216.5K ops/sec 822ns 822ns
Scan Throughput 100 2.958s 0.3 ops/sec 2.786s 3.298s
Scan Throughput 1000 22.405s 0.0 ops/sec 22.030s 22.764s
Typical Queue Shuffle 5.33µs 187.6K ops/sec 5.02µs 129.22µs
Priority Queue Shuffle 25.97µs 38.5K ops/sec 24.88µs 108.24µs

🐍 Python Version 3.11.15

@N7WEra

N7WEra commented May 4, 2026 via email

Copy link
Copy Markdown
Contributor

@shart123456

Copy link
Copy Markdown
Contributor

Code review

Found 1 issue:

  1. dnsresolve.py sets main_host_event.always_emit_tags = [] (no underscore prefix), but BaseEvent.always_emit reads from self._always_emit_tags (with underscore prefix) — the assignment is a no-op and has no effect on event emission behavior.

self.debug(
f"Skipping DNS children for {event} because their DNS resolve distances would be greater than the configured value for this scan ({self.helpers.dns.runaway_limit})"
)
main_host_event.add_tag(f"runaway-dns-{dns_resolve_distance}")
else:

The check in base.py that this is intended to override:

return self._always_emit or always_emit_tags or no_host_information
@property
def id(self):
"""
A uniquely identifiable hash of the event from the event type + a SHA1 of its data

The fix is either to use the correct underscore-prefixed name (_always_emit_tags) or to set main_host_event._always_emit = False directly.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

Comment thread bbot/test/test_step_1/test_presets.py Dismissed
@codecov

codecov Bot commented May 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.03137% with 580 lines in your changes missing coverage. Please review.
✅ Project coverage is 90%. Comparing base (370102e) to head (e3baa4a).
⚠️ Report is 33 commits behind head on stable.

Files with missing lines Patch % Lines
bbot/modules/virtualhost.py 73% 122 Missing ⚠️
bbot/core/helpers/web/web.py 77% 72 Missing ⚠️
bbot/modules/waf_bypass.py 80% 33 Missing ⚠️
bbot/modules/lightfuzz/submodules/crypto.py 83% 29 Missing ⚠️
bbot/modules/http.py 91% 27 Missing ⚠️
bbot/cli.py 63% 25 Missing ⚠️
bbot/core/event/base.py 85% 21 Missing ⚠️
bbot/core/modules.py 92% 21 Missing ⚠️
bbot/modules/lightfuzz/submodules/sqli.py 80% 19 Missing ⚠️
bbot/modules/gowitness.py 82% 16 Missing ⚠️
... and 43 more
Additional details and impacted files
@@           Coverage Diff            @@
##           stable   #3079     +/-   ##
========================================
- Coverage      91%     90%     -0%     
========================================
  Files         441     450      +9     
  Lines       37720   46100   +8380     
========================================
+ Hits        34122   41362   +7240     
- Misses       3598    4738   +1140     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread bbot/test/conftest.py Dismissed
@liquidsec

Copy link
Copy Markdown
Collaborator

recheck

liquidsec and others added 17 commits June 9, 2026 20:49
The SPF mechanism prefix regex moves to regexes.py (compiled with the regex package like the rest). IP/CIDR extraction now only runs on records containing v=spf1, and SPF macro tokens are skipped. DNS child events no longer hardcode DNS_NAME: an IP was coerced anyway, but a CIDR failed validation and was silently dropped, so the type is now auto-detected (DNS_NAME / IP_ADDRESS / IP_RANGE). Adds an end-to-end test from TXT record to emitted IP_ADDRESS / IP_RANGE events, plus macro and gating coverage.
CodeQL's incomplete-url-substring-sanitization query flags hostname string literals in membership asserts; these are exact set-membership checks, so switch the three flagged asserts to subset comparisons.
…secrets

Upgrade baddns to 2.4, badsecrets to 1.1
forward_event() distributes events to modules via sequential awaits.
Between iterations, a fast module can process the event and call
_minimize(), dropping _module_consumers to 0 and stripping fields
like original_value before slower modules are even queued.

Add a sentinel (+1 before the loop, _minimize() after) so the
counter never transiently hits 0 during distribution. Also use
defensive .get() in lightfuzz metadata() as a belt-and-suspenders
guard.
…in-forward-event

Fix _minimize() race in forward_event stripping WEB_PARAMETER fields
Shared helper on WebHelper detects hosts that return the same response
regardless of URL path (SPAs, catch-all redirects, soft-404s). Probes
two distinct random paths + root via HttpCompare with auto-filtered
dynamic content. Cached per (scheme, host, port), 3 requests per host.

Consumers: wayback skips CDX query, webbrute skips fuzzing, excavate
suppresses URL_UNVERIFIED (other extractions still run).
- Crypto: skip digit-only values in keystream-reuse detection (they're
  decimal IDs misread as hex, not ciphertexts)
- Add nlbi_ (Incapsula load balancer) to parameter_blacklist_prefixes
Prevents speculate from expanding out-of-scope CIDRs (e.g. from SPF
records) into individual IP_ADDRESS events that no downstream module
can use. Children land at distance+1, so enumerating a range beyond
search_distance just produces unreachable pipeline churn.
liquidsec and others added 25 commits June 29, 2026 22:16
event.md linked to ../../scanning/events (outside docs root); correct to ../scanning/events.md to match other dev pages. Wire the orphaned webbrute deep-dive page into the Modules nav.
…/ruff-0.15.20

Bump ruff from 0.15.18 to 0.15.20
- Remove SCAN event network section (feature was never committed)
- Link blasthttp repo in HTTP internals bullet (mirrors blastdns)
- Fix build-system bullet that split across lines due to a leading +
- Break the Message Queues table into per-module sections (kafka,
  rabbitmq, nats, zeromq, websocket) with config examples
- Reword the http reassignment note in the 3.0 migration guide
- filter with is_global instead of is_private so CGNAT/reserved space (no public ASN) isn't looked up
- serialize asndb requests via NamedLock so concurrent callers don't stampede the rate-limited bbot.io API
asn: filter non-global IPs and serialize asndb lookups
Updates the requirements on [fastapi](https://github.com/fastapi/fastapi) to permit the latest version.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](fastapi/fastapi@0.115.5...0.139.0)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-version: 0.139.0
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
…/fastapi-gte-0.115.5-and-lt-0.140.0

Update fastapi requirement from <0.139.0,>=0.115.5 to >=0.115.5,<0.140.0
- _check_output_path: resolve paths before the containment check so ".." and escaping symlinks are rejected
- filter_event: match github.com by hostname instead of substring
- download_run_artifacts: sanitize artifact filename to a safe basename
Defense-in-depth: the registry-supplied tag is used unsanitized in the tarball
filename, so reject any path that resolves outside output_dir before writing.
Matches the pattern in github_workflows, postman_download, and gitdumper.
The symlink/hardlink rejection in unarchive._check_archive_safe() was applied
only to the tar branch. The zip/7z branch parsed the p7zip `Attributes` value
and rejected only when it started with "l", but p7zip prefixes a DOS attribute
block before the unix mode (e.g. "_ lrwxrwxrwx" for a symlink), so the check
never fired and no `Link = ` line is emitted for zip/7z symlinks. A symlink
stored in a downloaded zip/7z was therefore accepted and materialized on disk
by `7z x`.

Parse the trailing unix mode field of the `Attributes` value and reject when
its type flag is "l" (symlink) or "h" (hardlink), mirroring the tar branch's
`line[0] in ("l", "h")` test. Add a symlink_zip case to
TestUnarchiveTraversalCheck alongside the existing symlink_tar.

Signed-off-by: tonghuaroot <tonghuaroot@gmail.com>
A unix-symlink zip lists as " lrwxrwxrwx" on mainline 7-Zip, which the pre-fix
guard already caught, so the zip case only exercises the fix on p7zip. A
7z-native symlink lists with a DOS-attribute prefix ("A lrwxrwxrwx") that the
pre-fix guard missed, covering the fix on mainline 7-Zip (what CI runs) too.
Reject symlink entries in zip/7z archives in unarchive module
Fix path traversal in github_workflows output path
@liquidsec liquidsec requested a review from ausmaster July 7, 2026 18:22
@en0f en0f self-requested a review July 7, 2026 18:26
@liquidsec liquidsec merged commit 5355bd1 into stable Jul 8, 2026
33 of 34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.