Skip to content

[plan] Add end-to-end regression test: validation and push must operate on the same object #48999

Description

@github-actions

Objective

Add an end-to-end/integration-style regression test for create_pull_request that asserts the validated .patch artifact and the actually-pushed .bundle/commit contain the identical file set, across both the normal path and the merge-commit rewrite path.

Context

The root cause shared by both bugs reported in issue #48934 is: validation and push do not operate on the same object, so a passing validation does not reliably predict what lands on the remote. This issue tracks adding a regression test suite that would have caught both bugs, and will guard against future variants (e.g., any new code path that rebuilds/rewrites the commit before push).

Suggested Approach

  1. Add a test harness (in actions/setup/js test suite or equivalent Go integration tests in pkg/workflow) that:
    • Simulates a base branch with drift (extra commits added after the agent's checkout, including files matching excluded-files).
    • Simulates a shallow checkout forcing the merge-commit rewrite path.
    • Runs the full create_pull_request flow (patch validation + bundle/push synthesis).
    • Extracts the file list of the validated .patch and the file list of the final pushed commit (via git diff --cached --name-only on the synthesized commit, or by inspecting the .bundle).
    • Asserts the two file lists are identical (same set of paths).
  2. Include a variant for the non-rewrite path (full-depth checkout, no merge commit) to ensure the same invariant holds there too, catching the excluded-files/bundle bug independently of the rewrite bug.
  3. Wire this test into the existing test suite so it runs in CI (make test-unit / relevant Go or JS test target) and will fail if a future change reintroduces the discrepancy.
  4. This issue depends on the fixes in the other sub-issues from this plan being complete (or can be developed test-first / marked expected-fail until those land).

Files to Modify

  • New test file(s) under actions/setup/js/ and/or pkg/workflow/ test directories
  • Possibly a small test-only helper to extract file lists from a bundle or synthesized commit

Acceptance Criteria

  • New regression test(s) assert patch file set == pushed/bundle file set, for both rewrite and non-rewrite paths
  • Test(s) fail on the pre-fix code (validate by checking against a pre-fix commit, or documenting expected failure) and pass after the companion fixes land
  • Test(s) integrated into the standard test run (make test-unit or equivalent)

Generated by 📋 Plan Command · aut00 · 18.7 AIC · ⌖ 5.78 AIC · ⊞ 6.8K ·
Comment /plan to run again

  • expires on Jul 31, 2026, 2:14 PM UTC-08:00

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions