Add shared Playwright title smoke test by Copilot · Pull Request #57721 · github/gh-aw

Copilot · 2026-09-01T15:31:45Z

Adds a reusable browser test that verifies agents can read JavaScript-rendered content with Playwright across engine and sandbox runtime combinations.

Shared test
- Starts a loopback Node.js server from workflow steps.
- Generates a cryptographically random expected title.
- Injects the title client-side so it is absent from the HTML returned to curl.
- Enables Playwright and prompts the agent to compare the rendered title exactly.
Coverage
- Imports the test into Copilot, Codex, Claude, and Gemini smoke workflows.
- Covers default and Cloud Hypervisor sandbox runtimes.
Generated workflows
- Recompiles affected lock files.
- Retains SHA-pinned action references.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 14.2 AIC · ⌖ 8.6 AIC · ⊞ 8.9K · ◷
_{Comment /souschef to run again}

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "github.com"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 19.5 AIC · ⌖ 8.76 AIC · ⊞ 8.9K · ◷
_{Comment /souschef to run again}

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan · 2026-09-01T15:37:35Z

      "version": "v6.1.0",
      "sha": "55cc8345863c7cc4c66a329aec7e433d2d1c52a9"
    },
+    "actions/checkout@v6.0.2": {


@copilot fix any instance of v6 and upgrade to v7

Fixed in 54697af — updated the pre-agent-steps checkout in daily-multi-device-docs-tester.md from actions/checkout@v6.0.2 to actions/checkout@v7 and recompiled the lock file.

Copilot

🟡 Changes recommended

The host-bound loopback server is unreachable from Playwright running inside the AWF sandbox.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a reusable Playwright smoke test for JavaScript-rendered page titles across agent engines and sandbox runtimes.

Changes:

Adds a randomized loopback title-test server and validation prompt.
Imports the test into seven smoke workflows.
Regenerates workflow locks and synchronizes action pins.

File summaries

File	Description
`.github/workflows/shared/playwright-title-test.md`	Defines the shared test.
`.github/workflows/smoke-copilot.md`	Enables the test for Copilot.
`.github/workflows/smoke-copilot.lock.yml`	Regenerates Copilot workflow.
`.github/workflows/smoke-copilot-small.md`	Enables Cloud Hypervisor coverage.
`.github/workflows/smoke-copilot-small.lock.yml`	Regenerates small-model workflow.
`.github/workflows/smoke-codex.md`	Enables the test for Codex.
`.github/workflows/smoke-codex.lock.yml`	Regenerates Codex workflow.
`.github/workflows/smoke-claude.md`	Enables the test for Claude.
`.github/workflows/smoke-claude.lock.yml`	Regenerates Claude workflow.
`.github/workflows/smoke-gemini.md`	Enables Gemini/Cloud Hypervisor coverage.
`.github/workflows/smoke-agent-scoped-approved.md`	Adds scoped-agent coverage.
`.github/workflows/smoke-agent-scoped-approved.lock.yml`	Regenerates scoped-agent workflow.
`.github/workflows/smoke-test-tools.md`	Adds tool-container coverage.
`.github/workflows/smoke-test-tools.lock.yml`	Regenerates tool smoke workflow.
`.github/aw/actions-lock.json`	Adds the checkout action pin.
`pkg/actionpins/data/action_pins.json`	Synchronizes action-pin data.
`pkg/workflow/data/action_pins.json`	Synchronizes compiler pin data.

Review details

Files reviewed: 18/18 changed files
Comments generated: 1
Review effort level: Balanced

Copilot · 2026-09-01T16:20:01Z

+      const fs = require("node:fs");
+      const http = require("node:http");
+
+      const host = "127.0.0.1";


Fixed in 5024e3d — the server now binds to 0.0.0.0 instead of 127.0.0.1, host.docker.internal is added to the workflow's network.allowed list, and the agent is instructed to navigate to http://host.docker.internal:4173/ instead of 127.0.0.1:4173.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

gh-aw-bot · 2026-09-01T16:10:44Z

@copilot Please address the latest review feedback and failed check on this PR, refresh the branch if needed, and run the pr-finisher skill before handing it back to maintainers.

Failed check: Windows Integration — https://github.com/github/gh-aw/actions/runs/33528127033/job/99927141343
Latest reviewer feedback: pelikhan review and copilot-pull-request-reviewer changes requested/comments are present on the current PR state.
Branch refresh was requested by sous-chef for run https://github.com/github/gh-aw/actions/runs/33529395448.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 14.2 AIC · ⌖ 8.6 AIC · ⊞ 8.9K · ◷
_{Comment /souschef to run again}

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot · 2026-09-01T16:24:11Z

@copilot Please address the latest review feedback and failed check on this PR, refresh the branch if needed, and run the pr-finisher skill before handing it back to maintainers.

Failed check: Wi...

Addressed in 5024e3d: the Playwright title server now binds to 0.0.0.0, host.docker.internal is allowlisted in network.allowed, and the agent prompt navigates to http://host.docker.internal:4173/ so the AWF sandbox can reach the host-side server. Replied to both review threads.

The Windows Integration failure (run 33528127033) was a transient cache-restore abort (Failed to restore cache entry... The operation was aborted), not caused by this PR's changes — no code fix applies. make fmt/make lint/make test-unit/make recompile all pass locally on the current HEAD. Since my pushes don't retrigger CI, a maintainer will need to re-trigger checks (push/close-reopen/workflow_dispatch) to get a fresh green run.

github-actions · 2026-09-01T17:03:50Z

Looks great! 🚀 Thanks for adding the Playwright title smoke test. This is a well-structured contribution that extends the test infrastructure with a reusable shared test for browser automation coverage.

The implementation is solid:

Clear test mechanics (loopback server with injected title, Playwright verification)
Integrated across all smoke workflows (Copilot, Claude, Codex, Gemini)
Proper coverage for both default and Cloud Hypervisor sandbox runtimes
Workflow lock files properly regenerated with SHA pinning retained

This PR is ready for review!

Generated by ✅ Contribution Check · copilot · auto · 46.5 AIC · ⌖ 7.92 AIC · ⊞ 9.3K · ◷

gh-aw-bot · 2026-09-01T17:38:14Z

@copilot Please address the current review feedback on this PR, inspect the branch after the refresh request, and run the pr-finisher skill before handing it back to maintainers.

Latest reviewer feedback: copilot-pull-request-reviewer has changes recommended on the current PR state.
Branch refresh was requested by sous-chef for run https://github.com/github/gh-aw/actions/runs/33537945047.
Please summarize the fixes you made and any remaining blockers for maintainers when done.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "github.com"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 19.5 AIC · ⌖ 8.76 AIC · ⊞ 8.9K · ◷
_{Comment /souschef to run again}

Copilot · 2026-09-01T17:44:44Z

@copilot Please address the current review feedback on this PR, inspect the branch after the refresh request, and run the pr-finisher skill before handing it back to maintainers.

Latest reviewer ...

No new code changes needed — the referenced copilot-pull-request-reviewer "changes recommended" review is the same one from commit 00b592e that was already fixed in 5024e3d (server bound to 0.0.0.0, host.docker.internal allowlisted, agent navigates via host.docker.internal:4173). Both review threads have replies confirming the fix.

The firewall warning (github.com blocked) comes from the Sous Chef workflow's own run (33537945047), not from this PR's changes — this PR doesn't touch that workflow's network config, so it isn't actionable here.

Current HEAD (5024e3d) already reflects all fixes; make fmt/make lint/make test-unit/make recompile pass locally. Remaining blocker: CI must be re-triggered by a maintainer since my pushes don't retrigger workflow runs.

github-actions · 2026-09-01T23:48:20Z

🎉 This pull request is included in a new release.

Release: v0.88.0

Copilot AI and others added 3 commits September 1, 2026 15:19

Plan shared Playwright title smoke test

23779d2

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Add shared Playwright title smoke test

caa59ab

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Restore pinned checkout action resolution

00b592e

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI assigned Copilot and pelikhan Sep 1, 2026

Copilot finished work on behalf of pelikhan September 1, 2026 15:32

Copilot created this pull request from a session on behalf of pelikhan September 1, 2026 15:32 View session

Copilot AI requested a review from pelikhan September 1, 2026 15:32

pelikhan marked this pull request as ready for review September 1, 2026 15:34

Copilot AI balanced review requested due to automatic review settings September 1, 2026 15:34

Copilot started reviewing on behalf of pelikhan September 1, 2026 15:34 View session

pelikhan reviewed Sep 1, 2026

View reviewed changes

Copilot AI reviewed Sep 1, 2026

View reviewed changes

Copilot started work on behalf of pelikhan September 1, 2026 15:42 View session

Upgrade daily-multi-device-docs-tester checkout to v7

54697af

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan September 1, 2026 15:50

Copilot AI requested a review from pelikhan September 1, 2026 15:50

Copilot started work on behalf of gh-aw-bot September 1, 2026 16:11 View session

Fix Playwright title test server unreachable from AWF sandbox

5024e3d

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot finished work on behalf of gh-aw-bot September 1, 2026 16:24

Copilot AI requested a review from gh-aw-bot September 1, 2026 16:24

This was referenced Sep 1, 2026

[pr-sous-chef] pr-sous-chef run report #57526

Closed

[Contribution Check Report] Contribution Check — 2026-09-01 #57577

Closed

Copilot started work on behalf of gh-aw-bot September 1, 2026 17:38 View session

Copilot finished work on behalf of gh-aw-bot September 1, 2026 17:45

pelikhan merged commit 078cb57 into main Sep 1, 2026
34 checks passed

pelikhan deleted the copilot/add-playwright-test-workflow branch September 1, 2026 18:10

github-actions Bot mentioned this pull request Sep 2, 2026

[static-analysis] RGS-012: Secret Exfiltration via Outbound HTTP Request in shared/playwright-title-test.md (7 workflows) #57854

Closed

Copilot AI mentioned this pull request Sep 2, 2026

Suppress RGS-012 for shared Playwright loopback probe #57858

Merged

github-actions Bot mentioned this pull request Sep 2, 2026

[deep-report] Reclassify approve_workflow_run/push_to_pull_request_branch protected-files policy decline as non-failure #57879

Open

Navigation Menu

Uh oh!

Add shared Playwright title smoke test - #57721

Add shared Playwright title smoke test#57721
pelikhan merged 5 commits into
mainfrom
copilot/add-playwright-test-workflow

Copilot AI commented Sep 1, 2026 •

edited by github-actions Bot

Loading

Uh oh!

pelikhan Sep 1, 2026

Uh oh!

Copilot AI Sep 1, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Sep 1, 2026

Uh oh!

gh-aw-bot commented Sep 1, 2026

Uh oh!

Copilot AI commented Sep 1, 2026

Uh oh!

github-actions Bot commented Sep 1, 2026

Uh oh!

gh-aw-bot commented Sep 1, 2026

Uh oh!

Copilot AI commented Sep 1, 2026

Uh oh!

Uh oh!

github-actions Bot commented Sep 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Copilot AI commented Sep 1, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pelikhan Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Copilot AI Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

gh-aw-bot commented Sep 1, 2026

Uh oh!

Copilot AI commented Sep 1, 2026

Uh oh!

github-actions Bot commented Sep 1, 2026

Uh oh!

gh-aw-bot commented Sep 1, 2026

Uh oh!

Copilot AI commented Sep 1, 2026

Uh oh!

Uh oh!

github-actions Bot commented Sep 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Sep 1, 2026 •

edited by github-actions Bot

Loading