Skip to content

fix(tests): follow-ups from the Codex bounded-time tests review (#2169) #2174

Closed as not planned

Description

@devswha

Follow-up to #2169 (notes from the merge pass's review; none blocked).

  • Length-only quadratic cost cancels out (omo MEDIUM, server/codex.test.ts): the reference and the subject now have the same length, so a mutant quadratic in length alone passes at 1.02 (the fix(tests): measure what the code controls in three tests that failed on busy hosts #2150 version caught it at 77.8 vs 40). The doc comment states this limit; consider a second, length-scaled reference.
  • The large-answer case has ~2x headroom under heavy load (9.63 vs 20), with allocation and GC in its ratio.
  • On a coarse clock a regression takes minutes to fail (the regex mutant ran >5 min); the retry loop has no upper bound.
  • Two comments quote lower "as written" ratios than measured (~1.3x and ~4x vs 1.3-3.1 and 2.8-9.6); changelog line 7 is 115 columns.

Activity

  1. devswha commented on Oct 28, 2026

    @devswha
    OwnerAuthor

    Closing under the new follow-up bar: a follow-up issue is kept only for a reproduced, user-visible MEDIUM finding. The mutants show test limits, not a user regression or an observed CI flake. Below the follow-up bar: LOW/unreproduced; recorded on PR #2169 and in this issue for reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions