Skip to content

test: two timing-sensitive cases fail merge-queue runs on PRs that do not touch them #7643

Closed

Description

@kyle-sexton

No related issue: new flaky-test report.

Summary

Two timing-sensitive test cases failed merge-queue runs for PRs that do not touch their code. Each PR passed the same check on its own CI, and a re-queue merged it.

Evidence

Fix

Make both cases independent of runner speed: give the hung-test case a deterministic stand-in for the hang, and measure the linear-time case by its growth ratio rather than a wall-clock limit. Or widen their limits and record how much headroom each gets.

🤖 Generated with Claude Code

https://claude.ai/code/session_011Wn4wZmSz8fVYJLNgFG8Zz

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 16, 2026
  2. github-actions commented on Oct 16, 2026

    @github-actions
    Contributor

    Thanks for the report. I understand this as two separate flaky, timing-sensitive test cases that fail in merge-queue runs on PRs unrelated to their code, then pass on re-queue or on the PR's own CI:

    1. lane-launcher.test.sh case [259] ("a hung plugin test reads unknown...") failing in a merge-queue run for feat: move check plugins to commit-time gates and per-batch formatting #7616.
    2. A mod test ("runs each of the worst accepted extra patterns in time linear in the text length") timing out at 5083ms against a 5000ms budget, in a merge-queue run for feat(typos-format): report at commit time through a shared git-hook installer #7547.

    Case 2 looks like a duplicate of #7273, which already covers the same linear-time mask test timing out under a fixed wall-clock budget on a shared runner. Case 1 (the lane-launcher hung-test case) looks new and not yet filed elsewhere.

    It may be worth splitting this into a new issue scoped to just the lane-launcher case, since the fix approaches differ (deterministic stand-in for the hang vs. growth-ratio or wider timeout for the linear-time test).

    Automated triage. A maintainer reviews it.

  3. added
    priority: mediumReal value, no hard deadline; normal backlog flow.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 16, 2026
  4. kyle-sexton commented on Oct 18, 2026

    @kyle-sexton
    MemberAuthor

    Agent Brief

    Type: Bug
    Summary: Make the lane-launcher hung-plugin-test case independent of runner speed.

    Current behavior:
    The lane-launcher test case a hung plugin test reads unknown (both start and force-bash variants) expects mods: unknown (claude plugin test did not finish in 1 s) and depends on a real hang hitting a 1 s wall-clock limit. It failed in merge-queue run 38949926510 on a PR that does not touch the lanes skill.

    Desired behavior:
    The case drives the hung claude plugin test path through a deterministic stand-in (for example a stub that never exits, with the launcher's timeout made injectable for the test, or a stub that reports the timeout outcome directly), so the result does not depend on how fast the runner is. The launcher's production timeout and its unknown reading are unchanged.

    Key interfaces:

    • Lane launcher's plugin-test timeout: the value production uses stays the same; the test may override it.
    • The mods: unknown (claude plugin test did not finish in N s) reading.

    Acceptance criteria:

    • scripts/run-test-file.sh plugins/harness-ops/skills/lanes/scripts/lane-launcher.test.sh exits 0.
    • The hung-test case no longer relies on a real process outlasting a wall-clock limit of the production value (read from the diff).
    • The launcher's production timeout value is unchanged (read from the diff).
    • A harness-ops changelog fragment is added and scripts/check-changelog-fragments.sh exits 0.

    Out of scope:


    This was generated by AI during backlog shaping. Held, not admitted: this brief is drafted and checked, but the issue lane would stop on it today. It is admitted, by adding agent-ready, once the lane can run the skill-quality check scripts (Automation Lanes' PR #7912). Held under the owner's 2026-10-18 lane-narrowing course correction.

  5. added
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.
    and removed
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Oct 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.priority: mediumReal value, no hard deadline; normal backlog flow.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions