CI testing approach review: flake, waste, and when-to-run matrix - #1406
Merged
Conversation
Re-verified the five branches against the live remote before touching anything, which mattered: claude/clinical-kb-pwa-review-asi3wb and claude/dazzling-blackwell-f348d0 are already gone, deleted by someone with the permission this session lacks and surfaced only by `git remote prune`. The list is three, not five, and #108 said five. The remaining three are still at exactly the recorded HEADs, and the proof was re-run and strengthened. For each, tree(tip) == tree(merge-base) byte-for-byte, so the branch nets zero content change from where it forked and nothing on it is absent from main. That is stronger than either check the cleanup guide names: --cherry-pick still reports 13/6/4 patch-unique commits on them, which is the squash-merge false positive the guide warns about, and none is an ancestor of main, so an ancestry --merged check would miss them too. A completed `branch-cleanup` row keyed to its own HEAD is now appended for each, which is the step that had to happen while the objects still exist — resolveHead refuses a HEAD that is not a commit in the repository, so the reverse order is unexecutable. check:branch-review-ledger passes at 1272 records with union merge intact. The refs can now be deleted safely at any time. Deletion itself is still blocked and is not mine to unblock: `git push origin --delete` exits 1 with HTTP 403 from the session git proxy, with recentRelayFailures empty, so it is credential scope rather than a relay fault; and the GitHub MCP set exposes create_branch but no delete-branch capability. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012YRCXgX4AWZ579bKN6sk6b
Append a branch-review ledger entry for the ci-testing-approach review covering hosted CI failure patterns, flake sources, and scope/waste findings. Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>
|
This pull request has been ignored for the connected project Preview Branches by Supabase. |
Contributor
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 45 minutes Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
Comment |
4 tasks
Append a review-ledger row for the read-only maturity audit of recommended tests and packages at main HEAD. Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>
Codex raised a P2 on PR #1403: marking these HEADs `branch-cleanup` suppresses future inspection while the rows themselves report 13/6/4 patch-unique commits, and tree equality between tip and merge-base only proves the branch's changes cancel in its final snapshot, not that each unique patch reached main. The conclusion does not hold for these branches, and measuring it settles which it is. For all three, the merge-base is an ANCESTOR of main — so tree(tip) == tree(merge-base) means every byte at the tip exists in main's own history. The changes did not cancel; they landed. The branches merged main in after their own PRs were squash-merged, which is why their diffs went empty. Checked concretely on the largest: the feature commit's new files (src/components/document-viewer/use-document-view-density.ts, use-printable-disclosures.ts, tests/document-view-density.dom.test.tsx) are present on origin/main as byte-identical blobs, and the 13 "unique" commits are six merges of main plus seven commits whose content is on main. But the procedural half of the finding was right. My original rows recorded tree equality WITHOUT recording that the merge-base is an ancestor of main — and tree equality alone does not imply the content is on main, since a tip could match an off-main commit. The proof was sound and the record of it was not. These superseding rows carry the full chain. The original exact-scope rows are left in place, so hasCompletedCleanupReview still resolves them; the ledger is append-only and this adds evidence rather than revising history. Verified: ancestry measured for all three merge-bases (855aa29, 79d1c87, b274048), not inferred from the one I had checked. check:branch-review-ledger passes at 1282 records with union merge intact. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012YRCXgX4AWZ579bKN6sk6b
5 tasks
…2vcd8a Complete #108's prerequisite and correct its branch list
BigSimmo
marked this pull request as ready for review
July 30, 2026 04:45
BigSimmo
enabled auto-merge (squash)
July 30, 2026 04:46
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 061058c53e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…nto wt-1406-check-branch
The prior row recorded ref as the unresolved placeholder "HEAD" instead of the branch, so ledger:lookup could never match it by branch name. Appends a corrected row (append-only; the original stays as history) with ref=cursor/ci-testing-review-1bf5. Verified: ledger:lookup now reports ALREADY REVIEWED for this branch/head/scope. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Re9ERtQwJ82ErbAnahAhsa
BigSimmo
pushed a commit
that referenced
this pull request
Jul 30, 2026
Codex review on #1408 caught both, and the first reproduced live minutes later on #1406: pushing a bundled addition to an already-open PR mid-CI-run cancels the in-flight run under cancel-in-progress: true, so it doesn't save an invocation, it restarts one. Clarify to wait for settle or assemble commits before the first push. Second: git revert <sha> for a bundled item only holds pre-merge — a normal squash-merge folds commits into one and deletes the feature branch, so post-merge per-item rollback means reverting hunks of the squash commit by hand. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Re9ERtQwJ82ErbAnahAhsa
BigSimmo
added a commit
that referenced
this pull request
Jul 30, 2026
Every newtask/handoff cycle mints a dedicated branch+PR, so a single docs/ledger-append line pays the same required-CI bill as a large change. PR #1406 measured 437 PR-triggered CI runs in a ~3-day window with ~40% cancelled and ~12 Production-UI-hours burned on superseded runs. Adds a bundling rule to AGENTS.md scoped to independently low-risk, separately-revertible work (never RAG-ranking-surface or clinical-governance-gated changes), a rationale entry in docs/process-hardening.md, and a nudge in the newtask skill to check for bundle candidates before minting a new branch. Claude-Session: https://claude.ai/code/session_01Re9ERtQwJ82ErbAnahAhsa Co-authored-by: Claude <noreply@anthropic.com>
2 tasks
BigSimmo
added a commit
that referenced
this pull request
Jul 30, 2026
… PR babysitting (#1421) * issues: capture two CI/merge operational findings from PR babysitting #117: this repo's CI (on: pull_request with default types) doesn't retrigger on the draft-to-ready transition, only on opened/synchronize/reopened — a marked-ready PR can sit with a minimal check set until an actual new commit lands. #118: GitHub's update-branch API doesn't honor the merge=union .gitattributes driver on docs/branch-review-ledger.md, so it can 422 with a false conflict that a local git merge resolves cleanly. Both observed today on PR #1406. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Re9ERtQwJ82ErbAnahAhsa * Address Codex review: clarify ledger rows #125/#126 wording #125: spell out that ready_for_review must be added alongside the existing pull_request.types defaults, not replace them. #126: qualify the update-branch fallback push with the same explicit user confirmation the provider boundary already requires. * docs(issues): capture #1396's unaddressed physical-device chrome gate PR #1396 repeatedly declared physical-device Safari/PWA acceptance (docs/phone-chrome-physical-acceptance.md) as required before merge because headless Chromium cannot certify Safari chrome minimisation or cold-launch PWA paint, then merged with the checklist still blank. Also notes a related missing pre-paint/cold-load hydration test the same PR's review flagged but never filed. * style: fix table padding drift from the main merge npx prettier --write after merging main (9e2fe44) — a table cell width shifted during the merge and format:check would have caught it. --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: BigSimmo <BigSimmo@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Review-only PR: records the
ci-testing-approachledger entry for HEAD13c16cf07c854b50daa35a2ef2a2ea76d5e059e1. No product or CI config changes.Evidence base: last 500
CIworkflow runs (~3 days of high PR traffic), workflow/ci-change-scopeinspection,docs/testing.md/docs/process-hardening.md, and open ledger items#093#095#097#023.Evidence snapshot
release-browser-matrixskipped (runs30216361999,29700014531)PR required,PR policy,Gitleaks(correctly scoped)Clinical Governance Preflight
RAG impact
RAG impact: no retrieval behaviour change — ledger documentation only; no RAG source or eval code touched.