You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
orb(safety): SELFHOST_DEPLOYMENT_MODE=dry-run does not reach the executor — a 'posts nothing' instance still cancels CI, globally blacklists contributors, and writes fabricated merge outcomes #9130
SELFHOST_DEPLOYMENT_MODE=dry-run — the posture .env.selfhost.example:71 ships as the recommended
first-boot default — does not stop the action executor. It is enforced one layer too low: inside the
HTTP client, after every decision that depends on it has already been made.
forcedSelfhostMode has exactly two references in src/, both in the same file:
Line 718 is inside makeInstallationOctokit. The executor computes its own mode from a completely
different input set (src/settings/agent-execution.ts:38-42):
SELFHOST_DEPLOYMENT_MODE appears nowhere in it. So under dry-run the executor computes live and
runs the entire mutation path. mergePullRequest returns the synthetic { merged: true, sha: null }
(client.ts:682) without throwing, so the executor takes its success branch and proceeds to do real,
non-GitHub work:
consequence
site
writes a real pr_outcome row recording "merged" — and its if (existing) return probe makes that row absorbing, so the PR's true later outcome can never be recorded
fires Discord/Slack notifications with outcome: "merged"
agent-action-executor.ts:620-631
maybeEscalateModeration short-circuits only on args.mode !== "live" — mode is live — so it records a violation and can upsertGlobalContributorBlacklist: a permanent global ban written off a mutation that never happened
agent-action-executor.ts:671, :753-769
What we promise
.env.example:30-34 states the dry-run instance "posts NOTHING … so it can shadow the live cloud App
safely." What actually happens on a shadow instance: contributor CI is cancelled in the shadowed repo,
contributors are globally blacklisted, operators are paged about merges that never occurred, and the
calibration ledger is permanently poisoned with fabricated ground truth — which then feeds computeGateEval → isHoldOnly / isCloseHoldOnly and changes the real instance's behaviour.
Trigger
Any self-host started from the shipped .env.selfhost.example, on the first contributor PR that reaches
a merge or close disposition.
Impact
The instance-level kill switch does less than every document describing it claims, and its failure mode
writes durable, contributor-visible harm (cancelled CI, global bans) plus durable data corruption
(fabricated outcomes). A safety control that is trusted precisely when an operator is least sure of their
setup is the worst place for this gap.
Requirements
Fold forcedSelfhostMode(env) into resolveAgentActionMode — or introduce a single resolveEffectiveActionMode(env, settings) that every call site uses — so the instance switch is the first precedence term, not an HTTP-layer afterthought.
Make the synthetic merge return not look like success to the executor, so a suppressed mutation can
never drive downstream side effects.
Add an invariant test: with SELFHOST_DEPLOYMENT_MODE=dry-run, a full merge disposition produces zero
GitHub writes, zero outcome rows, zero notifications, and zero moderation escalations.
Test Coverage Requirements
99%+ patch coverage, branch-counted. Both arms of the instance-mode precedence, plus the four downstream
side effects asserted absent under dry-run.
Problem
SELFHOST_DEPLOYMENT_MODE=dry-run— the posture.env.selfhost.example:71ships as the recommendedfirst-boot default — does not stop the action executor. It is enforced one layer too low: inside the
HTTP client, after every decision that depends on it has already been made.
forcedSelfhostModehas exactly two references insrc/, both in the same file:Line 718 is inside
makeInstallationOctokit. The executor computes its own mode from a completelydifferent input set (
src/settings/agent-execution.ts:38-42):SELFHOST_DEPLOYMENT_MODEappears nowhere in it. So underdry-runthe executor computesliveandruns the entire mutation path.
mergePullRequestreturns the synthetic{ merged: true, sha: null }(
client.ts:682) without throwing, so the executor takes its success branch and proceeds to do real,non-GitHub work:
pr_outcomerow recording"merged"— and itsif (existing) returnprobe makes that row absorbing, so the PR's true later outcome can never be recordedagent-action-executor.ts:1055→outcomes-wire.ts:365-402POST /repos/{r}/actions/runs/{id}/cancel, killing the contributor's CI (rawtimeoutFetch, outside the octokit hook)agent-action-executor.ts:603-608→src/github/app.ts:612outcome: "merged"agent-action-executor.ts:620-631maybeEscalateModerationshort-circuits only onargs.mode !== "live"— mode is live — so it records a violation and canupsertGlobalContributorBlacklist: a permanent global ban written off a mutation that never happenedagent-action-executor.ts:671,:753-769What we promise
.env.example:30-34states the dry-run instance "posts NOTHING … so it can shadow the live cloud Appsafely." What actually happens on a shadow instance: contributor CI is cancelled in the shadowed repo,
contributors are globally blacklisted, operators are paged about merges that never occurred, and the
calibration ledger is permanently poisoned with fabricated ground truth — which then feeds
computeGateEval→isHoldOnly/isCloseHoldOnlyand changes the real instance's behaviour.Trigger
Any self-host started from the shipped
.env.selfhost.example, on the first contributor PR that reachesa merge or close disposition.
Impact
The instance-level kill switch does less than every document describing it claims, and its failure mode
writes durable, contributor-visible harm (cancelled CI, global bans) plus durable data corruption
(fabricated outcomes). A safety control that is trusted precisely when an operator is least sure of their
setup is the worst place for this gap.
Requirements
forcedSelfhostMode(env)intoresolveAgentActionMode— or introduce a singleresolveEffectiveActionMode(env, settings)that every call site uses — so the instance switch is thefirst precedence term, not an HTTP-layer afterthought.
mode !== "live"guards that exist to compensate; they areindividually blind to this switch (
src/review/visual/actions-fallback.ts:106checks a mode that neversees
SELFHOST_DEPLOYMENT_MODE— the orb(safety): visual-capture workflow_dispatch bypasses the write-suppression chokepoint and fires under dry_run #9067 fix has the same blindness).src/github/app.ts:612) through the suppression chokepoint; it is the lastremaining installation-scoped write on raw
timeoutFetchand a direct sibling of orb(safety): visual-capture workflow_dispatch bypasses the write-suppression chokepoint and fires under dry_run #9067.never drive downstream side effects.
SELFHOST_DEPLOYMENT_MODE=dry-run, a full merge disposition produces zeroGitHub writes, zero outcome rows, zero notifications, and zero moderation escalations.
Test Coverage Requirements
99%+ patch coverage, branch-counted. Both arms of the instance-mode precedence, plus the four downstream
side effects asserted absent under dry-run.
Links & Resources
src/settings/agent-execution.ts~38-42;src/github/client.ts~682, ~693, ~718;src/services/agent-action-executor.ts~276, ~593-631, ~671, ~753-769, ~1055;src/github/app.ts~612;.env.selfhost.example~71;.env.example~30-34precedence for
agentDryRun, a different switch). Distinct root cause: the instance switch lives onelayer below every decision that depends on it.
Boundaries
Mode precedence and the executor's side-effect gating. No change to what
dry_runmeans or to theper-repo
agentDryRunswitch.maintainer-only — kill-switch architecture.