Skip to content

Migrate single-timeout ReDoS checks to scaling-ratio methodology across 37 languages #1071

Description

@squid-protocol

Issue

The shared harness tests/extraction/languages/_strict_harness.py provides two ReDoS test methods:

  1. assert_redos_immune (weak): Single payload, single 1.0s timeout. Can miss O(n²) backtracking that stays under 1.0s at the tested payload size.
  2. _best_of_timing (strong): Geometric-growth scaling ratio across multiple payload sizes (4x increase per doubling). The signature of real O(n²) backtracking — mandated by how_to_add_a_language.md's Strict Testing Prompt, Step 4, item 5: "measure actual execution time at several geometrically increasing sizes... a roughly 4x increase per doubling is the signature of real O(n²) backtracking."

Current state: Only 8 of 45 languages use _best_of_timing (css, c, cpp, csharp, groovy, makefile, tcl, yacc). The other 37 rely solely on the weaker single-timeout check across 392 total assert_redos_immune calls.

This is a detection-strength gap: the 37 languages run risk of shipping undetected ReDoS vulnerabilities at scale, especially in rules with the "adjacent-quantifier-overlap" shape (Rule 14 in how_to_add_a_language.md) which can blow up dramatically faster than a generous single timeout would ever catch.

Solution

  1. For each of the 37 languages currently using only assert_redos_immune:

  2. Consult tests/extraction/languages/_strict_harness.py and existing c/cpp/csharp _strict.py usage for the method signature and expected test patterns.

Files to modify

All tests/extraction/languages/test_<lang>_strict.py files except: css, c, cpp, csharp, groovy, makefile, tcl, yacc (these 8 are already migrated).

Related issues

Activity

  1. added
    testingUnit, integration, and E2E pipeline verification
    core-engineModifications to the central physics and parsing engine
    on Aug 5, 2026
  2. squid-protocol commented on Aug 5, 2026

    @squid-protocol
    OwnerAuthor

    Correction to the original framing, found while writing tests/extraction/how_to_harden_strict_signatures.md: this is not a mechanical "replace every assert_redos_immune call with _best_of_timing" migration. Looking at the one existing reference example (test_css_strict.py's test_css_class_start_redos_regression), the two helpers serve different moments:

    • _best_of_timing is used to reconstruct and prove an old, already-fixed pattern's quadratic scaling signature — a regression anchor, not a general-purpose test style.
    • assert_redos_immune proves the current pattern survives a large adversarial payload within a timeout — this is the correct, sufficient permanent test when there's no known bug.

    The actual task here: run the geometric-sweep diagnostic (verify_candidates.py's check_redos_scaling, or _best_of_timing called directly) across each of the 37 languages' quantifier-heavy rules to check whether a single generous 1.0s timeout is masking an undiscovered O(n²) bug. Most will come back clean (ratio ~2x per doubling) — for those, no change needed, assert_redos_immune alone is already correct. Only where the sweep finds real quadratic scaling (~4x per doubling) does it need a regex fix plus the two-part _best_of_timing-old-pattern + assert_redos_immune-new-pattern regression test.

    Full methodology now in tests/extraction/how_to_harden_strict_signatures.md's "A/B — ReDoS coverage" section.

  3. squid-protocol commented on Aug 6, 2026

    @squid-protocol
    OwnerAuthor

    Closed via PR #1083. Our comprehensive geometric sweep up to N=128,000 proved that the existing single-timeout assert_redos_immune tests are already mathematically sound and no underlying O(n^2) ReDoS bugs are being masked. We've committed the scanner tool used to prove this (sweep_redos_scaling.py) so the methodology is permanently documented in the git history.

  4. added this to the v2.4.8 milestone on Aug 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    core-engineModifications to the central physics and parsing enginetestingUnit, integration, and E2E pipeline verification

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions