Skip to content

Rewrite vhost module as virtualhost with native blasthttp#2993

Merged
liquidsec merged 24 commits into
devfrom
add-virtualhost-module
Jun 16, 2026
Merged

Rewrite vhost module as virtualhost with native blasthttp#2993
liquidsec merged 24 commits into
devfrom
add-virtualhost-module

Conversation

@liquidsec

@liquidsec liquidsec commented Mar 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

Complete rewrite of the vhost module (now renamed to virtualhost). The old module was a thin wrapper around the ffuf Go binary for Host header fuzzing. The new module is a native Python implementation using blasthttp for all HTTP requests, with significantly expanded detection capabilities.

Requires #2992 (blasthttp integration). Supersedes #2672.

What changed from the old vhost module

Architecture:

  • No longer inherits from ffuf or shells out to any external binary
  • All HTTP requests go through self.helpers.request() with resolve_ip for DNS pinning
  • Uses blasthttp cert_info for certificate SAN extraction
  • Response comparison via simhash instead of ffuf size/word/line filters

New detection capabilities:

  • Wildcard detection via single-character hostname mutation
  • Canary consistency re-verification before emitting results
  • Keyword-based wildcard filtering
  • Certificate SAN extraction via blasthttp cert_info
  • Interesting default content reporting
  • HTTPS virtual host support via SNI control with resolve_ip
  • Configurable simhash similarity threshold (default 0.8)

Event changes:

  • Produces VIRTUAL_HOST (was VHOST), DNS_NAME_UNVERIFIED, and HTTP_RESPONSE
  • Detailed descriptions with status code, title, size, IP, accessibility

Renamed: vhost -> virtualhost, VHOST -> VIRTUAL_HOST, deadly flag removed

@liquidsec liquidsec changed the title Add virtualhost module for virtual host discovery Rewrite vhost module as virtualhost with native blasthttp Mar 27, 2026
Brute-forces virtual hosts via HTTP Host headers and HTTPS SNI,
with wildcard detection, canary consistency checks, keyword-based
filtering, certificate SAN extraction, and mutation generation.
Uses blasthttp resolve_ip for DNS pinning.
@liquidsec liquidsec force-pushed the add-virtualhost-module branch from 07dfd28 to fecac39 Compare March 27, 2026 18:49
@github-actions

github-actions Bot commented Mar 27, 2026

Copy link
Copy Markdown
Contributor

🚀 Performance Benchmark Report

⚠️ No current benchmark data available

This might be because:

  • Benchmarks failed to run
  • No benchmark tests found
  • Dependencies missing

@codecov

codecov Bot commented Mar 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.94624% with 140 lines in your changes missing coverage. Please review.
✅ Project coverage is 90%. Comparing base (3b43e43) to head (1f3adf1).
⚠️ Report is 30 commits behind head on dev.

Files with missing lines Patch % Lines
bbot/modules/virtualhost.py 73% 122 Missing ⚠️
...est_step_2/module_tests/test_module_virtualhost.py 98% 12 Missing ⚠️
bbot/core/event/base.py 70% 6 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff           @@
##             dev   #2993    +/-   ##
======================================
- Coverage     90%     90%    -0%     
======================================
  Files        449     451     +2     
  Lines      44649   45577   +928     
======================================
+ Hits       39972   40766   +794     
- Misses      4677    4811   +134     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

liquidsec added 14 commits April 3, 2026 00:37
Move compute_simhash off the process pool (run_in_executor_mp) onto the
CPU thread pool (run_in_executor_cpu) via a new _response_similarity
helper. Simhash work is short and inputs are truncated to ~3KB inside
compute_simhash, so pickle/IPC overhead and broken-pool risk outweigh
any parallelism win — and the recent 300s timeout on run_in_executor_mp
was orphaning _test_virtualhost tasks with uncaught TimeoutError.

Also remove stale 'CurlError'/'curl error' debug strings left over from
the curl-helper removal.
resolved_hosts can include CNAME targets (hostnames) alongside A/AAAA
records, so the previous code emitted hostnames in the VIRTUAL_HOST
event's 'ip' field. Filter to actual IPs at the probe-candidate sites,
and use blasthttp's response.peer_ip (with host_ip fallback) for the
emitted ip — that's the ground-truth IP the response came from.
Prevents the rejection from bubbling up as a warning through as_completed.
@liquidsec liquidsec changed the base branch from blasthttp-integration-clean to dev May 12, 2026 18:08
@ausmaster ausmaster added this to the BBOT 3.0 - blazed_elijah milestone May 20, 2026
@liquidsec liquidsec mentioned this pull request Jun 15, 2026
28 tasks
Register VIRTUAL_HOST as DictHostEvent with host+virtual_host dedup,
use local RNG instead of global random.seed(), fix produced_events
(DNS_NAME_UNVERIFIED not DNS_NAME), drop unused pyOpenSSL dep, fix
config fallback values, and discard results on flood detection.
@ausmaster ausmaster self-requested a review June 16, 2026 20:59
@liquidsec liquidsec merged commit c346ead into dev Jun 16, 2026
15 checks passed
@liquidsec liquidsec deleted the add-virtualhost-module branch June 16, 2026 21:22
@liquidsec liquidsec mentioned this pull request Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants