Skip to content

Flaky test: codex.test.ts bounded-time matcher cases time wall-clock milliseconds on a loaded runner #2142

Closed

Description

@devswha

server/codex.test.ts "reads short answers in bounded time however many links a large answer has" (line 858 on c5d9555) fails on loaded CI runners with no change to the matcher:

Expected: < 1000
Received: 1062.94534
(fail) Codex rollout resolution > reads short answers in bounded time however many links a large answer has [1070.72ms]

Seen in the Fast checks job of PR runs for #2136, #2114 and #2098 (1013, 1030 and 1062 ms; e.g. job 122546701308). A rerun passes.

The case times matchShortCodexAnswers with performance.now(). On P620 at load average ~100 the same call takes 190-290 ms of wall time, and its thread CPU time is the same, so the 1 s budget is not set by the matcher's own work: a busy runner stretches the clock while the work waits for a core. Its two siblings (bounds citation parsing with many unmatched backtick lengths, ... however many unclosed links the answers hold) use the same wall-clock pattern.

Suggested fix: keep the 1 s bound but take it from this thread's CPU time (clock_gettime(CLOCK_THREAD_CPUTIME_ID), as src/lib/markdown.test.ts does since #2084), and show that it still fails when cutAtLinkTargets goes back to a quadratic scan.

Activity

  1. added a commit that references this issue on Oct 28, 2026
    99547b0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions