Skip to content

Graceful cloudcheck error handling + SSL verify passthrough#3239

Merged
liquidsec merged 1 commit into
devfrom
cloudcheck-ssl-graceful
Jun 25, 2026
Merged

Graceful cloudcheck error handling + SSL verify passthrough#3239
liquidsec merged 1 commit into
devfrom
cloudcheck-ssl-graceful

Conversation

@liquidsec

Copy link
Copy Markdown
Collaborator

Summary

Fixes #3226 — cloudcheck signature-fetch failures (notably SSL verification issues behind corporate proxies) were crashing scans at cli.py:237. This forwards bbot's SSL setting to cloudcheck and makes the remaining failure paths non-fatal.

Changes

  • SSL passthrough (core/helpers/helper.py): construct CloudCheck(verify_ssl=...) from ssl_verify_infrastructure, mirroring how asn.py wires the same setting into asndb. Requires the new verify_ssl kwarg, so the dependency is bumped to cloudcheck>=11.1.0,<12.
  • Graceful setup() (modules/internal/cloudcheck.py): the warm-up lookup now catches CloudCheckError, warns, and continues. Bucket regexes (loaded from the cloudcheck package) and per-event lookups (already guarded) keep working, so cloud detection degrades instead of failing setup.
  • Graceful pre-scan check (cli.py): the interactive "is this target a cloud domain?" lookup catches CloudCheckError and warns instead of aborting the scan.
  • Deps: cloudcheck 10.0.0 → 11.1.0, baddns → 2.4.1; uv.lock reconciled to the pyproject pins (this also moves ruff to the already-pinned 0.15.18).

Notes on the three requirements in #3226

  1. SSL passthrough — done (above).
  2. Fallback to cached data rather than raising — covered by cloudcheck's existing on-disk cache: after a successful fetch it reuses ~/.cache/cloudcheck/cloud_providers_v3.json for 24h without touching the network. The only uncovered sliver is a >24h-stale cache where the refresh then also fails (i.e. the network is genuinely down) — that now degrades gracefully via Two new modules, bugfixes and optimizations #3 rather than crashing. No bundled copy is shipped (the signature JSON is ~2.3 MB and isn't part of the package).
  3. Don't crash the scan — done: setup() and cli.py:237 are wrapped, scan continues with a warning.

Catches the specific CloudCheckError rather than a broad except.

Forward bbot's ssl_verify_infrastructure setting to cloudcheck's new
verify_ssl constructor param, and stop cloudcheck signature-fetch
failures from crashing scans.

- helper.py: CloudCheck(verify_ssl=ssl_verify_infrastructure)
- cloudcheck module setup(): catch CloudCheckError on the warm-up
  lookup; warn and continue with cloud detection degraded instead of
  aborting the scan
- cli.py: catch CloudCheckError on the pre-scan cloud-domain check
- bump cloudcheck 10.0.0 -> 11.1.0 (verify_ssl support) and baddns to
  2.4.1; lock reconciled to pyproject pins (also moves ruff to the
  already-pinned 0.15.18)

Closes #3226
@github-actions

Copy link
Copy Markdown
Contributor

📊 Performance Benchmark Report

Comparing dev (baseline) vs cloudcheck-ssl-graceful (current)

📈 Detailed Results (All Benchmarks)

📋 Complete results for all benchmarks - includes both significant and insignificant changes

🧪 Test Name 📏 Base 📏 Current 📈 Change 🎯 Status
Bloom Filter Dns Mutation Tracking Performance 4.32ms 4.28ms -1.0%
Bloom Filter Large Scale Dns Brute Force 17.71ms 17.71ms -0.0%
Large Closest Match Lookup 348.79ms 333.25ms -4.5%
Realistic Closest Match Workload 181.23ms 178.04ms -1.8%
Event Memory Medium Scan 1402 B/event 1402 B/event +0.0%
Event Memory Large Scan 1527 B/event 1527 B/event +0.0%
Event Validation Full Scan Startup Small Batch 400.27ms 397.35ms -0.7%
Event Validation Full Scan Startup Large Batch 511.07ms 508.43ms -0.5%
Make Event Autodetection Small 20.27ms 19.85ms -2.1%
Make Event Autodetection Large 204.33ms 202.22ms -1.0%
Make Event Explicit Types 8.34ms 8.36ms +0.2%
Excavate Single Thread Small 3.685s 3.569s -3.1%
Excavate Single Thread Large 9.335s 9.330s -0.0%
Excavate Parallel Tasks Small 3.822s 3.840s +0.5%
Excavate Parallel Tasks Large 5.876s 5.832s -0.8%
Intercept Throughput Small 864.30ms 963.40ms +11.5% 🟡🟡 ⚠️
Intercept Throughput Medium 862.54ms 870.15ms +0.9%
Dns Throughput Quiet 2.493s 2.607s +4.5%
Dns Throughput Loaded 1.902s 1.711s -10.1% 🟢🟢 🚀
Dns Throughput Inherited 2.059s 2.164s +5.1%
Is Ip Performance 2.31ms 2.32ms +0.7%
Make Ip Type Performance 210.65µs 196.74µs -6.6%
Mixed Ip Operations 2.39ms 2.38ms -0.3%
Memory Use Web Crawl 1.0 MB 1.0 MB +0.0%
Memory Use Subdomain Enum 28.3 MB 28.3 MB +0.0%
Memory Use Deep Chain 5.0 MB 5.0 MB +0.0%
Memory Use Parallel Chains 11.5 MB 10.2 MB -11.0% 🟢🟢 🚀
Scan Throughput 100 2.627s 2.546s -3.1%
Scan Throughput 1000 17.650s 17.903s +1.4%
Typical Queue Shuffle 5.67µs 5.48µs -3.3%
Priority Queue Shuffle 27.27µs 27.06µs -0.8%

🎯 Performance Summary

+ 2 improvements 🚀
! 1 regression ⚠️
  28 unchanged ✅

🔍 Significant Changes (>10%)

  • Intercept Throughput Small: 11.5% 🐌 slower
  • Dns Throughput Loaded: 10.1% 🚀 faster
  • Memory Use Parallel Chains: 11.0% 🚀 less memory

🐍 Python Version 3.11.15

@codecov

codecov Bot commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 38.46154% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 90%. Comparing base (3bc8e9c) to head (5727446).

Files with missing lines Patch % Lines
bbot/cli.py 0% 6 Missing ⚠️
bbot/modules/internal/cloudcheck.py 60% 2 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff          @@
##             dev   #3239   +/-   ##
=====================================
- Coverage     90%     90%   -0%     
=====================================
  Files        453     453           
  Lines      46304   46314   +10     
=====================================
+ Hits       41411   41413    +2     
- Misses      4893    4901    +8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ausmaster ausmaster added this to the BBOT 3.0 - blazed_elijah milestone Jun 25, 2026
@ausmaster ausmaster self-requested a review June 25, 2026 19:35
@liquidsec liquidsec merged commit 15e884c into dev Jun 25, 2026
17 checks passed
@liquidsec liquidsec deleted the cloudcheck-ssl-graceful branch June 25, 2026 19:50
@liquidsec liquidsec mentioned this pull request Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants