Don't fail safe_outputs job when a resolve-review-thread ID hits an unrelated repo by Copilot · Pull Request #53124 · github/gh-aw

Copilot · 2026-08-16T13:00:43Z

The safe_outputs job in PR Sous Chef failed because a resolve_pull_request_review_thread call was made with a thread ID that resolved to an unrelated repository (juju/juju), tripping the allowed-repos check. Since the workflow never configured cross-repo resolution, this was almost certainly a stale or fabricated GraphQL node ID — but the handler treated the mismatch as a fatal error, failing the entire job even though four other safe outputs in the same run had already succeeded.

Handler behavior

resolve_pr_review_thread.cjs: in legacy (default, non-cross-repo) mode, a thread resolving to a disallowed repo is now marked skipped: true instead of a hard failure. The action still isn't performed (fail-closed is preserved), but a single bad ID no longer takes down the whole safe_outputs job — same treatment as already-resolved/stale threads. Explicit cross-repo misconfigurations (via target/allowed-repos) still fail fatally, since those indicate real setup errors.

// legacy mode: thread resolves to an unrelated repo
core.warning(`Thread ${resolvedThreadId} repository ${threadRepo} is not allowed in legacy mode; skipping`);
return {
  success: false,
  skipped: true, // was: fatal failure
  thread_id: resolvedThreadId,
  error: legacyRepoValidation.error,
};

Prompt hardening

pr-sous-chef.md: instructions now explicitly require copying PRRT_... thread IDs verbatim from reviewThreads output rather than guessing, truncating, or extending them, to reduce the chance of malformed IDs being submitted in the first place.

Tests

Updated resolve_pr_review_thread.test.cjs to assert the legacy cross-repo mismatch case is now skipped rather than fatal.
Added coverage in safe_output_handler_manager.test.cjs confirming a skipped resolve_pull_request_review_thread result is excluded from fatalFailures while still being counted in the overall skip/failure status (not silently dropped).

Fixes [aw] Failed jobs: PR Sous Chef #53094

Run: https://github.com/github/gh-aw/actions/runs/31950301208> Generated by 👨‍🍳 PR Sous Chef · gpt54 · 12.5 AIC · ⌖ 5.77 AIC · ⊞ 8.7K · ◷

_{Comment /souschef to run again}

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 8.45 AIC · ⌖ 6.8 AIC · ⊞ 6.3K · ◷
_{Comment /souschef to run again}

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 7.61 AIC · ⌖ 5.75 AIC · ⊞ 8.7K · ◷
_{Comment /souschef to run again}

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 14.4 AIC · ⌖ 6.04 AIC · ⊞ 8.7K · ◷
_{Comment /souschef to run again}

github-actions · 2026-08-16T13:11:37Z

Hi @pelikhan 👋 — thanks for tasking the Copilot bot with this workflow fix! I see this is a work-in-progress (draft) PR addressing the failed safe_outputs job in the PR Sous Chef workflow (closes #53094).

Since this PR is still in draft with no code changes yet, here are the things to watch for as the implementation progresses:

Add concrete changes — Once the fix is ready, make sure the diff clearly addresses the root cause of the safe_outputs job failure.
Keep description updated — As noted in the PR, continue updating the description with progress and context so reviewers understand the fix strategy.
Consider test coverage — If the fix involves workflow logic or YAML changes, consider adding or updating tests if the project has workflow test patterns.

The PR is properly on-topic and follows the core-team contribution process. Once code is added, we can assess focus and completeness more fully. Looking forward to seeing the fix!

Generated by ✅ Contribution Check · auto · 56.8 AIC · ⌖ 3.74 AIC · ⊞ 9.1K · ◷

…_outputs job Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

…ropped Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

github-actions · 2026-08-16T13:32:06Z

✅ Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR does not have the implementation label and has 0 new lines of code in business logic directories (threshold: 100).

🏗️ ADR gate enforced by Design Decision Gate 🏗️

github-actions · 2026-08-16T13:32:12Z

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff

🧪 Test quality analysis by Test Quality Sentinel

github-actions · 2026-08-16T13:32:19Z

✅ Ponytail Reviewer completed successfully!

Lean already. Ship. Reviewed PR #53124 diff for over-engineering; changes are minimal and targeted (added skipped flag, warning message tweak, one prompt-hardening line, matching test updates). Nothing to cut.

Generated by Ponytail Reviewer for #53124

github-actions · 2026-08-16T13:32:37Z

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

github-actions · 2026-08-16T13:32:59Z

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

github-actions · 2026-08-16T13:36:41Z

Comment Memory

reviewed_at: 2026-08-16T00:00:00Z
review_event: COMMENT
top_themes:
  - no actionable blocking issues in changed lines
  - tests cover skipped-vs-fatal handler classification
files_reviewed:
  - .github/workflows/pr-sous-chef.md
  - actions/setup/js/resolve_pr_review_thread.cjs
  - actions/setup/js/resolve_pr_review_thread.test.cjs
  - actions/setup/js/safe_output_handler_manager.test.cjs
comment_count: 0

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · gpt54 · 4.81 AIC · ⌖ 5.44 AIC · ⊞ 4.5K · ◷
_{Comment /review to run again}

github-actions

Verdict

Non-blocking from this pass: the changed lines look internally consistent, and the new skipped-vs-fatal behavior is covered by targeted tests.

Highlights

The handler still fails closed by refusing the cross-repo resolution, but now marks the legacy-mode mismatch as skipped so one bad thread ID does not sink the entire safe_outputs job.
The manager test covers the important accounting edge case: skipped thread-resolution results are excluded from fatal failures while still counting toward skipped item status.
I did not find a changed-line regression that clearly warrants blocking merge.

🔎 Code quality review by PR Code Quality Reviewer · gpt54 · 4.81 AIC · ⌖ 5.44 AIC · ⊞ 4.5K
_{Comment /review to run again}

Copilot

Pull request overview

Prevents unrelated review-thread IDs from failing the entire safe_outputs job while preserving repository access restrictions.

Changes:

Treats legacy-mode repository mismatches as skipped operations.
Hardens PR Sous Chef instructions against fabricated thread IDs.
Adds coverage for handler and failure-partitioning behavior.

Show a summary per file

File	Description
`actions/setup/js/resolve_pr_review_thread.cjs`	Skips disallowed legacy-mode thread IDs.
`actions/setup/js/resolve_pr_review_thread.test.cjs`	Tests the new skip behavior.
`actions/setup/js/safe_output_handler_manager.test.cjs`	Verifies skipped results remain non-fatal.
`.github/workflows/pr-sous-chef.md`	Requires verbatim thread IDs.
`.github/workflows/pr-sous-chef.lock.yml`	Updates the compiled workflow metadata hash.

Review details

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Files reviewed: 5/5 changed files
Comments generated: 0
Review effort level: Balanced

github-actions

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd — the fix is targeted, well-reasoned, and backed by good tests. Approving with one minor observation.

📋 Key Themes & Highlights

Key Themes

Root cause vs. symptom: The fix correctly identifies that a legacy-mode repo mismatch almost always means a hallucinated/stale ID, not a genuine cross-repo access attempt. Downgrading to skipped rather than fatal is the right call.
Fail-closed preserved: Explicit cross-repo misconfigurations (via target/allowed-repos) still fail fatally — the safety boundary is maintained.
Test coverage: Both unit files are updated: the handler test asserts skipped: true, and the manager test confirms skipped results flow into itemsSkipped rather than fatalFailures.

Positive Highlights

✅ isFailedProcessingResult already gates on classifySafeOutputResult === "failed", so skipped: true results are automatically excluded from fatalFailures without a special-case in partitionFailureResults.
✅ Prompt hardening instruction is precise and actionable.
✅ Test names read as specifications.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 · 29.4 AIC · ⌖ 8.36 AIC · ⊞ 7.7K
_{Comment /matt to run again}

github-actions

The fix is correct and well-tested.

The skipped: true flag on the legacy cross-repo result correctly routes through classifySafeOutputResult → "skipped" → excluded from partitionFailureResults fatal failures. Tests cover both the handler-level and manager-level behavior. The companion workflow instruction to copy thread IDs verbatim is a sensible preventive addition.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · sonnet46 · 25.6 AIC · ⌖ 7.94 AIC · ⊞ 5.6K

gh-aw-bot · 2026-08-16T13:43:03Z

@copilot PR #53124 looks ready for a final pass.

Quick triage

Recent checks visible on the current head SHA look green.
I didn't find an active sous-chef cooldown comment on this PR.
Please run the pr-finisher skill to do the final merge-readiness sweep and address anything it surfaces.

Next step

If the PR is truly ready, finish the branch and move it toward merge.

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 12.5 AIC · ⌖ 5.77 AIC · ⊞ 8.7K · ◷
_{Comment /souschef to run again}

gh-aw-bot · 2026-08-16T14:38:54Z

@copilot PR #53124 looks ready for a final pass.

Quick triage

Recent checks visible on the current head SHA look green.
I did not find an active sous-chef cooldown comment on this PR.
Please run the pr-finisher skill to do the final merge-readiness sweep and address anything it surfaces.

Next step

If the PR is truly ready, finish the branch and move it toward merge.

Run: https://github.com/github/gh-aw/actions/runs/31953024314

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 7.61 AIC · ⌖ 5.75 AIC · ⊞ 8.7K · ◷
_{Comment /souschef to run again}

gh-aw-bot · 2026-08-16T14:51:48Z

@copilot PR #53124 looks ready for a final pass.

Quick triage

Recent checks visible on the current head SHA look green.
I did not find an active sous-chef cooldown comment on this PR.
Please run the pr-finisher skill to do the final merge-readiness sweep and address anything it surfaces.

Next step

If the PR is truly ready, finish the branch and move it toward merge.

Run: https://github.com/github/gh-aw/actions/runs/31953623248

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 14.4 AIC · ⌖ 6.04 AIC · ⊞ 8.7K · ◷
_{Comment /souschef to run again}

github-actions · 2026-08-18T22:24:59Z

🎉 This pull request is included in a new release.

Release: v0.87.1

Initial plan

38c0fd2

Copilot AI assigned Copilot and pelikhan Aug 16, 2026

Copilot started work on behalf of pelikhan August 16, 2026 13:00 View session

Copilot AI linked an issue Aug 16, 2026 that may be closed by this pull request

[aw] Failed jobs: PR Sous Chef #53094

Closed

github-actions Bot mentioned this pull request Aug 16, 2026

[Contribution Check Report] Contribution Check Report — 2026-08-16 #53004

Closed

Copilot AI and others added 2 commits August 16, 2026 13:13

Prevent stale/hallucinated PR review thread IDs from failing the safe…

625b416

…_outputs job Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Address code review feedback: assert skipped result is counted, not d…

452b1e3

…ropped Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix failed jobs in PR Sous Chef workflow~~ Don't fail safe_outputs job when a resolve-review-thread ID hits an unrelated repo Aug 16, 2026

Copilot finished work on behalf of pelikhan August 16, 2026 13:19

Copilot AI requested a review from pelikhan August 16, 2026 13:19

pelikhan marked this pull request as ready for review August 16, 2026 13:31

Copilot AI balanced review requested due to automatic review settings August 16, 2026 13:31

Copilot started reviewing on behalf of pelikhan August 16, 2026 13:31 View session

github-actions Bot reviewed Aug 16, 2026

View reviewed changes

github-actions Bot mentioned this pull request Aug 16, 2026

[aw] No-Op Runs #48838

Open

Copilot AI reviewed Aug 16, 2026

View reviewed changes

github-actions Bot approved these changes Aug 16, 2026

View reviewed changes

Merge branch 'main' into copilot/fix-failed-jobs-pr-sous-chef

ce85a25

github-actions Bot mentioned this pull request Aug 16, 2026

[pr-sous-chef] PR Sous Chef run report #52990

Closed

Copilot started work on behalf of gh-aw-bot August 16, 2026 13:43 View session

Copilot finished work on behalf of gh-aw-bot August 16, 2026 13:55

Copilot AI requested a review from gh-aw-bot August 16, 2026 13:55

Merge branch 'main' into copilot/fix-failed-jobs-pr-sous-chef

f090379

Copilot started work on behalf of gh-aw-bot August 16, 2026 14:39 View session

Copilot finished work on behalf of gh-aw-bot August 16, 2026 14:48

Copilot started work on behalf of gh-aw-bot August 16, 2026 14:52 View session

Copilot finished work on behalf of gh-aw-bot August 16, 2026 14:56

pelikhan merged commit 5b1ffe4 into main Aug 16, 2026
2 checks passed

pelikhan deleted the copilot/fix-failed-jobs-pr-sous-chef branch August 16, 2026 16:29

github-actions Bot mentioned this pull request Aug 16, 2026

Smoke Test: MAI-Code-1-Flash - 31974724676 #53212

Closed

Uh oh!

Conversation

Copilot AI commented Aug 16, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Handler behavior

Prompt hardening

Tests

Uh oh!

github-actions Bot commented Aug 16, 2026

Uh oh!

github-actions Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 16, 2026

Comment Memory

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Verdict

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Skills-Based Review 🧠

Key Themes

Positive Highlights

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

gh-aw-bot commented Aug 16, 2026

Quick triage

Next step

Uh oh!

gh-aw-bot commented Aug 16, 2026

Quick triage

Next step

Uh oh!

gh-aw-bot commented Aug 16, 2026

Quick triage

Next step

Uh oh!

Uh oh!

github-actions Bot commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Aug 16, 2026 •

edited by github-actions Bot

Loading

github-actions Bot commented Aug 16, 2026 •

edited

Loading

github-actions Bot commented Aug 16, 2026 •

edited

Loading

github-actions Bot commented Aug 16, 2026 •

edited

Loading

github-actions Bot commented Aug 16, 2026 •

edited

Loading

github-actions Bot commented Aug 16, 2026 •

edited

Loading