collapse PTR-style floods in chaos response#3115
Merged
Merged
Conversation
Chaos sometimes returns more than a million subdomains for a single domain, typically ISP reverse-DNS explosions. Without intervention this floods the event pipeline. - Drop subdomains whose leading segments look like IPv4 octets (dotted or hyphenated), the most common PTR-record shape. - After filtering, group surviving names by immediate parent. If a parent has more than PER_PARENT_CAP (300) children, drop them and emit just the parent — let bbot's normal DNS wildcard detection decide whether it actually is wildcard, rather than asserting it from chaos's evidence alone. - Run the filter/group/collapse in the CPU executor so processing large responses doesn't block the event loop. Resolves #2879.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
🚀 Performance Benchmark Report
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## dev #3115 +/- ##
=======================================
+ Coverage 90% 91% +1%
=======================================
Files 447 448 +1
Lines 40462 42611 +2149
=======================================
+ Hits 36320 38355 +2035
- Misses 4142 4256 +114 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This was referenced May 20, 2026
- Skip per-parent cap when parent == query (apex), so >300 direct-apex subdomains are not silently dropped - Require exactly 4 groups of 1-3 digits in PTR patterns, preventing false positives on date-prefixed names - Add TestChaosApexFlood and date-survival assertion
Deduplicate children per parent (set instead of list) so duplicates from the chaos API don't inflate the per-parent cap count and silently drop legitimate subdomains.
ausmaster
approved these changes
Jun 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Chaos sometimes returns over a million subdomains for a single domain — typically ISP reverse-DNS explosions where every customer IP has a PTR record under the same parent. Today bbot pushes every one of those names through the pipeline, causing huge memory use and effectively DoSing the scan.
001.106.103.218.fooand000-1-246-220.foo.PER_PARENT_CAP(300) children, drop them and emit just the parent. We deliberately do NOT assert wildcard ourselves — bbot's standard DNS wildcard detection runs on the parent and rewrites the event to_wildcard.<parent>if it actually is wildcard. If it isn't (e.g. ISP PTRs that all resolve to distinct IPs), we just lose the noisy children, which we couldn't usefully enumerate anyway.run_in_executor_cpu) so processing >1M string entries doesn't block the event loop.PER_PARENT_CAPis set to 300 after looking at the parent-size distribution on a known noisy domain: the histogram is bimodal — parents are either small (≤49 children, real subdomain clusters) or massive (≥462 children, PTR explosions). 300 is the safe middle.Resolves #2879.