Skip to content

CI: timestamp the browser lane's test server lines and explain the empty fork-PR failures #897

Closed

Description

@devswha

Split from #552: main already prints per-script timings in scripts/ci-browser.sh and keeps screenshots from the browser lane on failure (#627).

Remaining

  • Timestamps on the test servers' own lines. The web UI servers that the browser scripts start log to the lane's output. Their lines are mixed with the PASS lines and carry no timestamps of their own. Add timestamps so a server line can be placed against a PASS line. The frames the page received are also not kept today; herdr's test-server.log is the only server log uploaded on failure.
  • Empty CI failures of fork PRs. Seven CI runs of fork PRs on 2026-10-06 ended as failure with no jobs and no log: 37455493445, 37452965707, 37443957584, 37437006600, 37436949778, 37411882568, 37411593274. They are not test failures. The likely cause is an approval-gated run that was closed or superseded, but the run metadata does not say. Look into them once, so the CI workflow's failure count can be read without subtracting them by hand.

No retries: a failure that a retry turns green is the kind this is meant to explain.

Activity

  1. devswha commented on Oct 23, 2026

    @devswha
    OwnerAuthor

    Looked into the second item (the empty CI failures of fork PRs). The first item, timestamps on the browser lane's lines, merged in #1789.

    What the seven runs are. Each is a pull_request run of an earlier commit of a fork PR that never started: 0 jobs, a check suite with 0 check runs, run_attempt 1. GitHub concluded it failure in the same second the PR was closed or merged, long after it was created:

    Run PR Run created PR closed/merged Run concluded
    37455493445 #515 (merged) 10-06 11:18:12 12:07:27 12:07:28
    37452965707 #514 (merged) 10-06 10:55:15 12:16:06 12:16:07
    37411593274 #504 (merged) 10-06 03:59:49 07:39:55 07:39:56
    39221321874 #1380 (closed) 10-19 08:14:09 10-20 01:49:39 10-20 01:49:40

    What that means. These runs sat waiting, most likely for the approval GitHub requires before a fork PR's workflow runs. The PR's later commits each got their own run, which ran and passed (#515: 37459272358 success; #514: 37459746193 and 37461135205 success; #504: 37430165753 success). When the PR closed, GitHub ended the old waiting runs as failure.

    What it is not. It is not a broken workflow file: actionlint passes on ci.yml at those runs' commits (26e5896, f74042f, 8188793). It is not a test failure either: no job ever started.

    So when the CI workflow's failures are counted, a pull_request run with conclusion: failure and 0 jobs, finished at its PR's close time, is one of these and can be left out. It still happens: #1380's run above is from 2026-10-19.

  2. devswha commented on Oct 25, 2026

    @devswha
    OwnerAuthor

    Fixed by #1789 (96eff68): the browser lane now timestamps its test server's output (scripts/ci-browser-parallel.ts, scripts/ci-browser.sh), and the empty fork-PR failures are explained in the comment above (zero jobs, concluded when the PR closed). Shipped in v0.4.14.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions