Repository navigation
Migrate single-timeout ReDoS checks to scaling-ratio methodology across 37 languages #1071
Description
Issue
The shared harness tests/extraction/languages/_strict_harness.py provides two ReDoS test methods:
assert_redos_immune(weak): Single payload, single 1.0s timeout. Can miss O(n²) backtracking that stays under 1.0s at the tested payload size._best_of_timing(strong): Geometric-growth scaling ratio across multiple payload sizes (4x increase per doubling). The signature of real O(n²) backtracking — mandated byhow_to_add_a_language.md's Strict Testing Prompt, Step 4, item 5: "measure actual execution time at several geometrically increasing sizes... a roughly 4x increase per doubling is the signature of real O(n²) backtracking."
Current state: Only 8 of 45 languages use _best_of_timing (css, c, cpp, csharp, groovy, makefile, tcl, yacc). The other 37 rely solely on the weaker single-timeout check across 392 total assert_redos_immune calls.
This is a detection-strength gap: the 37 languages run risk of shipping undetected ReDoS vulnerabilities at scale, especially in rules with the "adjacent-quantifier-overlap" shape (Rule 14 in how_to_add_a_language.md) which can blow up dramatically faster than a generous single timeout would ever catch.
Solution
-
For each of the 37 languages currently using only
assert_redos_immune:- Identify rules with adjacent-quantifier overlaps or nested/unbounded character classes (scan for patterns like
*\*,+\+,[^\]]*, etc.) - Priority order: rules already flagged as Re DoS risks in epic Epic: Strict & exact regex test coverage for all structural signatures, per language #518 findings (e.g., the
spec_exposurecopy-paste bug class across 28 languages from issue spec_exposure's unbounded [^\]]* ReDoS is copy-pasted across 28 languages, including already-merged yacc #713). - Convert at least the top 3 most-complex rules per language from
assert_redos_immuneto_best_of_timingwith pathological multi-size payloads.
- Identify rules with adjacent-quantifier overlaps or nested/unbounded character classes (scan for patterns like
-
Consult
tests/extraction/languages/_strict_harness.pyand existingc/cpp/csharp_strict.pyusage for the method signature and expected test patterns.
Files to modify
All tests/extraction/languages/test_<lang>_strict.py files except: css, c, cpp, csharp, groovy, makefile, tcl, yacc (these 8 are already migrated).
Related issues
- Epic Epic: Harden the strict structural-signature test suite (non-extraction-pillar rules) per language #1069: Harden the strict structural-signature test suite (non-extraction-pillar rules) per language
- Issue spec_exposure's unbounded [^\]]* ReDoS is copy-pasted across 28 languages, including already-merged yacc #713: spec_exposure's unbounded [^]]* ReDoS is copy-pasted across 28 languages
- Epic Epic: Strict & exact regex test coverage for all structural signatures, per language #518: Strict & exact regex test coverage for all structural signatures, per language (CLOSED)
Activity
squid-protocol commentedon Aug 5, 2026
Correction to the original framing, found while writing
tests/extraction/how_to_harden_strict_signatures.md: this is not a mechanical "replace everyassert_redos_immunecall with_best_of_timing" migration. Looking at the one existing reference example (test_css_strict.py'stest_css_class_start_redos_regression), the two helpers serve different moments:_best_of_timingis used to reconstruct and prove an old, already-fixed pattern's quadratic scaling signature — a regression anchor, not a general-purpose test style.assert_redos_immuneproves the current pattern survives a large adversarial payload within a timeout — this is the correct, sufficient permanent test when there's no known bug.The actual task here: run the geometric-sweep diagnostic (
verify_candidates.py'scheck_redos_scaling, or_best_of_timingcalled directly) across each of the 37 languages' quantifier-heavy rules to check whether a single generous 1.0s timeout is masking an undiscovered O(n²) bug. Most will come back clean (ratio ~2x per doubling) — for those, no change needed,assert_redos_immunealone is already correct. Only where the sweep finds real quadratic scaling (~4x per doubling) does it need a regex fix plus the two-part_best_of_timing-old-pattern +assert_redos_immune-new-pattern regression test.Full methodology now in
tests/extraction/how_to_harden_strict_signatures.md's "A/B — ReDoS coverage" section.squid-protocol commentedon Aug 6, 2026
Closed via PR #1083. Our comprehensive geometric sweep up to N=128,000 proved that the existing single-timeout
assert_redos_immunetests are already mathematically sound and no underlying O(n^2) ReDoS bugs are being masked. We've committed the scanner tool used to prove this (sweep_redos_scaling.py) so the methodology is permanently documented in the git history.test(extraction): add geometric redos sweeping tool to verify strict …
docs: draft PR and Epic notes for #1071
chore: clean up scratch/debug debris and dedupe fortran deep-case blo…
test(extraction): deepen strict structural-signature coverage across …
Docs testing updates (#1091)
docs: add language-status skill and docs/language_status/ (Python wor…
docs: add cobol language-status doc (sections 1-8) (#1893)