Skip to content

Run the bats suite on main pushes (coverage parity with unit) — re-scoped after #1485 correction #1486

Closed

Description

@don-petry

Story

As a maintainer relying on main as the source of truth for agent scripts,
I want the bats suite to run on pushes to main with the same coverage it has on PRs,
so that trunk-state regressions in shell-script behavior are surfaced by every merge, matching the parity unit already has.

Problem (re-scoped 2026-08-08 — see the correction comment on #1485)

Original framing claimed both corrupting merges in #1485 landed with "zero post-merge signal." That was wrong: test-dev-lead.yml (unit) does run on main pushes (paths-filtered) and ran green on both corrupting merges — the signal existed but was blind to duplicate-definition corruption (bash last-definition-wins). The structural guard for that failure mode is #1482, not more test runs.

What remains true and worth fixing: the bats job lives in lint.yml, which triggers on pull_request and workflow_dispatch only. ci.yml (push+PR) covers lint/security but not bats. So shell-script behavioral coverage from the bats suites exists on PRs but never re-runs against actual trunk state after a squash merge — a gap unit does not have.

Acceptance Criteria

  1. The bats job (or an equivalent job reusing scripts/run-bats.sh) runs on push: branches: [main], paths-filtered like its PR trigger (scripts/**, tests/**, .github/workflows/**).
  2. No change to PR-triggered behavior; no new required checks on PRs.
  3. A failing main-push run is visible to the Fleet Monitor (confirm main-branch failures for lint.yml appear in its reports, as petry-projects — workflow failures detected 2026-08-08 #1484 suggests they do; otherwise add an on-failure issue-open step).
  4. template-drift remains excluded from any main-push wiring until [bug] seed-repo-template fetches standards unpinned — pin to published stable, compliance accepts N-1 (continues #1436) #1448 is resolved.

Tasks

  • Add the paths-filtered push: branches: [main] trigger to lint.yml (or split the bats job into its own workflow if lint-on-main-push is undesirable — keep the diff minimal either way).
  • Verify one green main-push run after merge.

Dev Notes

References

Activity

  1. changed the title [-]Run unit/bats suites on main pushes — trunk merged corrupted twice with zero post-merge signal[/-] [+]Run the bats suite on main pushes (coverage parity with unit) — re-scoped after #1485 correction[/+] on Aug 8, 2026
  2. don-petry commented on Aug 18, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: issue #1486 implementation failed — will retry

    The engine failed (engine-error — e.g. a timeout or transient engine error). This was attempt 1 of 3; dev-lead will retry automatically. You can also re-apply the dev-lead label to retry now.

    Last 40 lines of engine output (redacted)
        [copilot] calling gh copilot (model=openai/o4-mini, timeout=2700s, flags=--yolo)
    Error: Classic Personal Access Tokens (ghp_) are not supported by Copilot.
    
    The GH_TOKEN environment variable contains a classic PAT.
    Please use a Fine-Grained Personal Access Token or another authentication method.
    
    To fix this, you can:
      • Replace the token in GH_TOKEN with a fine-grained PAT
      • Unset GH_TOKEN and run 'gh auth login' to authenticate
      • Unset GH_TOKEN and start 'copilot', then use the '/login' command
    
    
  3. don-petry commented on Aug 18, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #1486 — Run the bats suite on main pushes (coverage parity with unit) — re-scoped after #1485 correction

    Scope

    Add a paths-filtered push: branches: [main] trigger to lint.yml so the bats job re-runs against trunk state after every merge, and exclude the known-red template-drift job from main-push runs (#1448) — no change to PR-triggered behavior.

    Implementation Checklist

    Tests to Write

    • tests/test_lint_main_push_trigger.bats — assert lint.yml has push: branches:[main] with the three paths, that the pull_request trigger is unchanged, and that template-drift carries the event_name != 'push' guard

    Test Command

    scripts/run-bats.sh tests/test_lint_main_push_trigger.bats (full suite: the bats job's file list in lint.yml)

    AC3 note

    Fleet Monitor already tracks lint.yml failures across all events/branches (fleet_monitor.sh queries actions/workflows/<id>/runs with no event filter, and lint.yml is not in FLEET_GATE_WORKFLOWS), so a failing main-push run surfaces in its reports with no extra wiring — matching #1484. No on-failure issue-open step needed.

  4. don-petry commented on Aug 18, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Implementation Complete

    Plan Execution

    • Added paths-filtered push: branches: [main] trigger to .github/workflows/lint.yml on: block, scoped to scripts/**, tests/**, .github/workflows/** (AC1)
    • Added if: github.event_name != 'push' to the template-drift job so it stays excluded from main-push wiring until [bug] seed-repo-template fetches standards unpinned — pin to published stable, compliance accepts N-1 (continues #1436) #1448 is resolved (AC4)
    • Left the pull_request + workflow_dispatch triggers and PR paths untouched — no new required PR checks (AC2)
    • Wrote tests/test_lint_main_push_trigger.bats guard and registered it in the bats job's file list
    • Confirmed AC3 needs no wiring (see Notes)

    Test Results

    1..5
    ok 1 lint.yml declares a push trigger scoped to the main branch (AC1)
    ok 2 lint.yml main-push trigger is paths-filtered to the bats-relevant paths (AC1)
    ok 3 lint.yml still triggers on pull_request — PR behavior unchanged (AC2)
    ok 4 lint.yml pull_request paths are unchanged by the push wiring (AC2)
    ok 5 template-drift is excluded from main-push runs until #1448 is resolved (AC4)
    

    Repo lint (.dev-lead/scripts/dev-lead-lint.sh): all checks passed (shellcheck on 110 files + agent-profile/CODEOWNERS validation).
    scripts/validate-workflow-schedules.sh: OK — no minute-0 crons introduced.
    yamllint (ci.yml config) on lint.yml: passes (the sole comment-indentation warning at the caller-stub-freeze block pre-exists on main and is untouched).

    Files Changed

    Notes

  5. added a commit that references this issue on Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dev-leadFor dev-lead agent pickupenhancementFeature requestsinitiativeEpic / initiative tracking issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions