Skip to content

Stop the Playwright suite failing on a fork default and an unfinished shard - #116

Open
gordlin wants to merge 1 commit into
masterfrom
gordon/ci-health
Open

gordlin wants to merge 1 commit into
masterfrom
gordon/ci-health

Conversation

@gordlin

@gordlin gordlin commented Sep 18, 2026

Copy link
Copy Markdown

How the feature works? / How did you fix the issue?

Two changes that stop the Playwright suite reporting failures that are not about the
code under test. Every recent PR fails a large batch of specs, which makes the suite
useless as a gate — telling a real regression from the noise currently takes a manual
run-to-run diff.

The S-Group specs assume a default this fork changed

1a41ba32e ("clear history stack") changed the S-Group dialog's default type:

- const type = ssgroup.type || 'DAT';
+ const type = ssgroup.type || 'SRU';

Upstream's specs open the type dropdown by clicking the span that shows the current
type, matched on the literal text 'Data'. On this fork that span reads "SRU polymer",
so the click waits the full 60 seconds and the test fails. That accounts for 168 of
the click timeouts CI reports.

The helpers now use the s-group-type-input-span test id, which this repo already uses
in is-query-structure-selected.spec.ts and s-group-click-atom-bond.spec.ts. They no
longer depend on which type is the default, so they work on the fork and would work
upstream. The text parameter every caller passed as 'Data' is gone with it.

Measured locally on multiple-s-group-tool.spec.ts:

before after
click timeouts 11 0
run time 3.3 min 25 s

Shard 6 of 6 has never finished

Master's last three runs and every PR run hit the 120-minute cap on part (6, 6) and
were cancelled without printing a summary. A sixth of the suite therefore has no history
at all, so its failures cannot be compared against anything — that is why #115's CI
looked far worse than master's when the two are in fact almost identical.

Ten shards bring each one back under an hour.

Test plan

  • multiple-s-group-tool.spec.ts locally, against a standalone build: 11 click timeouts
    to 0. The 11 remaining failures are A snapshot doesn't exist, from Playwright looking
    for -chromium-darwin.png on macOS; that cannot happen on Linux CI.
  • npx tsc --noEmit and eslint clean over the changed specs.
  • The first CI run on this branch is itself the check for the shard change.

Not included

About 150 screenshot comparison failures remain, concentrated in
toggle-explicit-hydrogens, image-files and calculate-cip-tool — the areas where the
fork deliberately changed what gets drawn (Indigo hydrogen label default, CIP
descriptors, S-group brackets) without regenerating upstream's committed snapshots.

Those need a scoped, reviewed regeneration, as a snapshot-only PR. Doing it before this
one lands would mean guessing at the list, since the current failure picture comes from
runs where a sixth of the suite never reported and every S-Group spec was timing out.

Check list

  • unit-tests written
  • e2e-tests written
  • documentation updated
  • PR name follows the pattern #1234 – issue name
  • branch name doesn't contain '#'
  • PR is linked with the issue
  • base branch (master or release/xx) is correct
  • task status changed to "Code review"
  • reviewers are notified about the pull request

🤖 Generated with Claude Code

…add shards

The fork changed the S-Group dialog's default type from Data to SRU, in
1a41ba3 "clear history stack". Upstream's specs open the type dropdown by
clicking the span that shows the current type, matched on the text "Data", so
every one of them waits 60 seconds and fails. That is 168 of the click timeouts
CI reports.

They now use the s-group-type-input-span test id, which this repo already uses
elsewhere and which does not depend on which type is the default. Locally the
Multiple S-Group file goes from 11 click timeouts to 0, and from 3.3 minutes to
25 seconds.

Shard 6 of 6 has never finished: master's last three runs and every PR run hit
the 120 minute cap and were cancelled without a summary, so a sixth of the suite
has no history to compare against. Ten shards bring each one back under an hour.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@gordlin gordlin self-assigned this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant