Skip to content

CI: Flag issue mentions that are not linked in the Development field - #77396

Open
adamsilverstein wants to merge 8 commits into
trunkfrom
add-development-releationship-ci-requirement-and-automation
Open

adamsilverstein wants to merge 8 commits into
trunkfrom
add-development-releationship-ci-requirement-and-automation

Conversation

@adamsilverstein

@adamsilverstein adamsilverstein commented Apr 15, 2026 •

Copy link
Copy Markdown
Member

Updated after the feedback on #77397 to cut down on noise and to stop nudging people into closing issues they only reference.

Claude wrote this description and kept it current through review:

What?

Closes #77397

Adds an advisory CI workflow, check-linked-issues.yml, that scans an open PR's description for issue references (#1234, GH-1234, or full issue URLs) and compares them against the PR's Development relationship (GraphQL closingIssuesReferences). If a referenced issue isn't linked, the workflow posts or updates a single comment explaining how to link it.

Why?

Based on Amy's feedback: anything included in a release should be linked (not just mentioned) so that it shows up in the Development section on the right sidebar of the related ticket. Today, tracking tickets like the per-release editor tasks ticket rely on release leads manually following every comment and every PR mentioned in every tracker to figure out what belongs in the release, and things still slip through. Surfacing the relationship in the sidebar lets release leads and release-note writers find every PR attached to a ticket at a glance.

The check is meant to be non-invasive:

  • It does not edit the PR body.
  • It does not fail the PR, it's advisory.
  • It only comments when unlinked mentions are detected, and removes its comment once they're linked.
  • Mentions that are clearly informational are skipped (see below), so "Related to" and "Part of" style PRs don't get a comment.

Linking an issue, whether with a closing keyword or from the sidebar, closes that issue when the PR is merged. The bot comment says so plainly and asks people to only link issues the PR fully resolves.

A more aggressive variant (auto-appending a bot-managed Closes #N block to the PR body) was considered but not chosen, because it would close issues that were only referenced for context.

How?

  • New workflow: .github/workflows/check-linked-issues.yml.
  • Runs on pull_request_target (opened, edited, reopened, synchronize, ready_for_review) so it reacts to description edits. It never checks out or runs PR code.
  • Runs are serialized per PR with a concurrency group, so two quick events can't both post a comment.
  • Uses actions/github-script (v9.0.0, matching the other workflows) to:
    1. Strip HTML comments, fenced code blocks and inline code from the body, so the template's Closes <!-- #ISSUE-NUMBER --> placeholder and values like color: #333 aren't read as issues.
    2. Extract referenced numbers from bare #N, GH-N, and in-repo issue/pull URLs.
    3. Skip a mention when the phrase directly before it marks it as informational (Related to #1, Part of #2, #3, See: #4, Follow-up to, Refs, cc, Reverts, Blocked by, Depends on, Supersedes, and similar), and skip WordPress core Trac tickets (Trac #62345).
    4. Query closingIssuesReferences for the issues already linked in this repository. Linked issues from other repositories are ignored, so a same-numbered issue elsewhere can't hide a mention here. GitHub resolves closing keywords asynchronously, so if anything still looks unlinked the check waits 10 seconds and asks once more.
    5. Check the remaining numbers in one batched GraphQL query and keep only real issues (not PRs, not missing numbers).
    6. Post or update a single comment (marked with an HTML-comment tag) listing unlinked issues, delete it once everything is linked, and remove any duplicates.
  • Permissions are issues: read and pull-requests: write on the default GITHUB_TOKEN. No additional secrets.
  • Gated to WordPress/gutenberg and open PRs only.
  • The script is annotated with step-by-step block comments (Step 1-10) to make the flow easy to follow.

Note: GitHub doesn't fire a workflow event when an issue is linked from the sidebar, so in that case the comment clears on the next push or description edit. The comment says this.

Testing Instructions

Test in WordPress Playground

  1. Open or edit a test PR that mentions an issue in the body without a closing keyword (eg. This addresses #100).
  2. Confirm the workflow runs and posts a comment listing the unlinked issue, including the warning that linking closes the issue on merge.
  3. Change the mention to Related to #100 and confirm the comment is removed.
  4. Change it to Closes #100 (or link the issue from the Development sidebar and then push or edit the body) and confirm the comment stays removed.
  5. Verify a PR description with no issue mentions produces no comment.
  6. Verify a PR description that only mentions PR numbers, Trac tickets, or #123 inside code produces no comment.

Follow-ups

AI Use

Claude Code wrote the code and this description, and I steered. I will review and test.

Add a non-invasive pull_request_target workflow that parses the PR body
for issue references (#N, GH-N, issue URLs), compares them against the
PR's closingIssuesReferences via GraphQL, and posts or updates a sticky
comment listing any referenced issues that aren't surfaced in the
Development sidebar. The comment suggests using a closing keyword or
manually linking the issue so release leads and release-note writers
can find every PR attached to a ticket. The check is advisory and does
not fail the PR.
@github-actions

github-actions Bot commented Apr 15, 2026 •

Copy link
Copy Markdown

Flaky tests detected in 9d4fe16.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/24481862957
📝 Reported issues:

@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown

🤖 PR meta 🤖

🎉 Props

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: adamsilverstein <adamsilverstein@git.wordpress.org>
Co-authored-by: kittenkamala <amykamala@git.wordpress.org>
Co-authored-by: Mamaduka <mamaduka@git.wordpress.org>
Co-authored-by: andrewserong <andrewserong@git.wordpress.org>
Co-authored-by: t-hamano <wildworks@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.


Props Bot commands

You can trigger Props Bot actions by commenting on this PR:

  • @propsbot add <username> to give props to a GitHub user. Requires write access.
  • @propsbot remove <username> to leave a user out of the props. Requires write access.
  • @propsbot refresh to refresh this list. Adding the props-bot label works too.

Updated as activity occurs, without notifying anyone named here. Add the props-bot label to refresh.

The workflow needs pull_request_target to comment on fork PRs, and
it never checks out or runs code from the PR.
@adamsilverstein

Copy link
Copy Markdown
Member Author

@desrosj appreciate your feedback here!

adamsilverstein and others added 4 commits November 8, 2026 09:18
Reviewers noted that many PRs reference issues with "Related to" or
"Part of" without resolving them, and that nudging contributors to link
those would close the issues on merge. Ignore mentions that follow an
informational phrase on the same line, and warn in the comment that
linking closes the issue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018vwYjUgqt8Ro82RJ8FLH1e
PR descriptions are soft-wrapped, so a whole paragraph sits on one line.
Matching "see" or "cc" anywhere earlier on the line hid real unlinked
mentions later in the paragraph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018vwYjUgqt8Ro82RJ8FLH1e
Fixes found in code review:

- Grant issues: read, since the job reads issues through REST and
  GraphQL.
- Ignore code blocks, inline code and Trac ticket numbers so CSS
  colours and core tickets aren't reported as Gutenberg issues.
- Serialize runs per PR and remove duplicate bot comments, so racing
  events can't leave a stale notice behind.
- Only count linked issues from this repository; a linked issue in
  another repo with the same number no longer hides a mention.
- Re-check links once after a short wait, since GitHub resolves closing
  keywords asynchronously.
- Look up all candidates in one GraphQL query instead of one REST call
  each.
- Note that sidebar links refresh the notice on the next push or edit.
- Use github-script v9.0.0 like the other workflows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018vwYjUgqt8Ro82RJ8FLH1e

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Type] Build Tooling Issues or PRs related to build tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: Surface when PR descriptions mention issues that are not linked in the Development field

2 participants