Skip to content

Epic: fixes to the current factory that are worth making whatever is rebuilt (lessons from the factory MVP review) #2276

Open

Description

@don-petry

Why this exists

The October 2026 architecture review of the agentic factory (petry-projects/.github-private#2225 — docs: factory MVP — PRD, architecture review, market research and ADR-0013 (proposed)) found things in the current system that are worth fixing whether or not the factory is rebuilt. In that pull request they are examples of lessons learned. This epic is where they get solved.

The review called the first two groups "worth having whatever is decided" and "the cheapest test of how much of the rest is needed". The maintainer wants a weekly comparison of the current system against the rebuilt one, so these changes are also what makes the current system's side of that comparison fair.

Evidence date. The review read the code at commit aef61164 on 2026-10-10. main has moved since. Every item below starts by re-checking it against main; two already look addressed and are marked.

A. Configuration fixes (review §8, step 0)

  • Update a pull request's branch only when it conflicts with its base. The merge queue tests the real merge result, so a branch does not need to be current. On 2026-10-11 .github/workflows/auto-rebase.yml still documents eligibility: all as the default. With 14 open pull requests, one merge to main starts about 784 jobs against 20 runner slots (review §14.1). Related, closed: petry-projects/.github-private#1881 — auto-rebase still merges main into every open PR on every push, though the ruleset no longer requires up-to-date branches — the treadmill's driver outlived the strict fix.
  • Try concurrency.queue: max on the dev-lead and pr-review lanes of one repo. GitHub now allows up to 100 pending runs in a concurrency group. No workflow here used it on 2026-10-11. Measure the share of runs cancelled before they start, before and after.
  • Pin the Claude Code CLI version. .github/workflows/dev-lead-reusable.yml resolves vars.CLAUDE_CODE_VERSION and falls back to latest. Set the variable. The CLI's documentation says its bare mode will become the default for headless runs and that bare mode does not read the subscription credential every agent here uses.
  • Make the retry and sweep schedules match what comments promise. Open: petry-projects/.github-private#2170 — Retry and sweep crons run every 4–8 hours instead of every 2 h / 15 min, while PR comments promise specific retry times.

B. Narrow what gates an approval (review §14.3)

The largest failure class in the issue history is the approval that never issues. The review found it is mostly policy: every comment from nine third-party reviewers needs a verified disposition.

  • Only a review gates. A review thread, or a review that requests changes, from a person with write access holds a pull request. A plain comment steers the next run and holds nothing.
  • Third-party reviewer output is input to the one review pass. It is never waited for and never gates by itself. This goes further than ADR-0012 (the comment gate's exemption set, proposed), which narrows the gate by marker.
  • Take Copilot and Qodo off the required set, and stop waiting for Codex. Maintainer decision of 2026-10-10: neither Copilot nor Qodo is reviewing, and Codex refuses for quota more often than it reviews.
  • Start a model review only after the deterministic checks have passed on that content.

Measure, weekly, before and after: the share of agent-authored merges with a standing approval, the median time from open to merge, and new issues about approvals that never issue.

C. Defects found in passing (review §12)

Concrete faults, each worth fixing on its own. File references are in the review.

  • The per-pull-request automation budget cannot count dev-lead, because its bot list omits the account dev-lead acts as.
  • AGENTS_PAUSED paused dev-lead only, while pr-review's error message told operators to use it. pr-review references the variable on main as of 2026-10-11; re-verify that it stops admission.
  • The fleet monitor's wake-up for dev-lead carries no pull-request or issue number, and the receiver skips a payload without one.
  • The CI failure analyst's prompt loses its run_id variable to unscoped envsubst.
  • Four bats files run in no workflow, and scripts/validate-engines.sh is wired to nothing.
  • The timeout guardrail checks the sum of all tier budgets against a job that runs one tier.
  • Token metering records only successful calls, and names the wrong model after a fallback in two tiers.
  • The proactive Claude headroom probe needs a key no workflow supplies, and logs "usage 0% — ok" on every call.
  • scripts/cut-release.sh does not know three registered agents.
  • The standards repo still dispatches to a workflow that was removed from this repo.
  • dev-lead fetches a gate script from another repo's default branch at run time and sources it, failing open on any error.
  • release/registry.yml is a second ring registry that only a test reads, and it disagrees with the real one.
  • Four documents contradict the code on live behaviour. CLAUDE.md no longer calls the repo private on main as of 2026-10-11; re-verify the other three.
  • The daily fleet failure report appears to have stopped after 2026-09-20.

How to work this

  • Each item becomes its own issue, with at most four acceptance criteria, before any agent label is added.
  • Release them to dev-lead one at a time.
  • This epic carries no agent label and should not be given one.

Not in scope

Anything in the new petry-projects/software-factory repo. This epic is about the system that is running today.

Activity

  1. github-actions commented on Oct 11, 2026

    @github-actions
    Contributor

    Thanks for putting together such a thorough epic and tying it back to the factory MVP review. Since the plan is to split each item into its own issue with at most four acceptance criteria, which item should go first, and is there a specific order you want them released to dev-lead beyond the A, B, C grouping?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions