Skip to content

Align compiler with gh-aw-firewall v0.28.6 (Cloud Hypervisor filesystem.allowWrite support) #55169

Description

@lpcox

Summary

gh-aw-firewall v0.28.6 ships full filesystem.allowWrite support for the Cloud Hypervisor microVM runtime. The compiler-side plumbing for allowWrite already exists in this repo but is inert by default, because DefaultFirewallVersion is still below the gate. This issue tracks aligning the compiler.

⚠️ This is not a simple version bump. Activating the gate as-is would very likely regress every Cloud Hypervisor workflow. See Blocker 1 below — please read before bumping.

Merged upstream PRs

Firewall PR Merge commit Content
#7660 6aab9a15 allowWrite planner (landed inert)
#7661 62de6370 Host mount-tree enforcement (landed inert)
#7669 664ffc1c Cloud Hypervisor runtime integration — makes the above live
#7672 4017c61c Critical fix — overlay mount propagation
#7670 5853b08b .awf-home prerequisite docs

All five are contained in tag v0.28.6 (fe3bcc3a801091efda4fb4a7902078a8fd0a9dfb).


Current compiler state

Already present, no changes needed to the mechanism itself:

  • pkg/constants/version_constants.go:134AWFFilesystemAllowWriteMinVersion = "v0.28.5"
  • pkg/workflow/awf_feature_flags.go:93awfSupportsFilesystemAllowWrite() gate
  • pkg/workflow/awf_config.go:548 — emits the filesystem section when gated, logs a skip otherwise

Blocking the feature:

  • pkg/constants/version_constants.go:67DefaultFirewallVersion = "v0.28.4" → below v0.28.5, so the gate never opens and the filesystem section is never emitted.

🔴 Blocker 1 — the default write path makes the CH workspace read-only

pkg/workflow/sandbox.go:286 (ensureDefaultAgentWritePath) unconditionally appends defaultAgentWorkspaceWritePath (/tmp/gh-aw/agent) to filesystem.allowWrite for every AWF sandbox config, regardless of runtime.

Under Docker/gVisor that is fine. Under Cloud Hypervisor it is not, because /workspace and /tmp/gh-aw are separate exports and the planner narrows each export independently. Since no allowed path falls under /workspace, that export gets narrowed to nothing.

I verified this empirically against the released v0.28.6 planner, passing exactly what ensureDefaultAgentWritePath produces today (allowWrite: ["/tmp/gh-aw/agent"]):

TAG=workspace   disposition=read-only  hostRoot=ro  guest=ro  overlays=0
TAG=tmp-gh-aw   disposition=selective  hostRoot=ro  guest=rw  overlays=1

Consequences if the version is bumped without addressing this:

  1. /workspace becomes fully read-only — the repo checkout is no longer writable, so the agent cannot edit files, and git operations that write to the worktree or .git fail.
  2. HOME becomes read-only — Cloud Hypervisor sets the guest home to /workspace/.awf-home, which is inside the now-read-only workspace export.

This is a silent, wide-blast-radius behavior change: it activates on the version bump alone, with no frontmatter change by any workflow author.

Suggested resolution — make the defaults runtime-aware. When the runtime is Cloud Hypervisor, ensureDefaultAgentWritePath should also seed the workspace paths the agent genuinely needs, at minimum:

  • /workspace (or a narrower set if the intent is to genuinely restrict the checkout), and
  • /workspace/.awf-home

Alternatively, skip emitting filesystem entirely for Cloud Hypervisor until the default set has been designed deliberately. Please treat "what should be writable by default under CH" as a product decision rather than inheriting the Docker default by accident.


🔴 Blocker 2 — .awf-home must exist on the host before AWF starts

The planner requires every allowWrite path to already exist, and Cloud Hypervisor does not create $GITHUB_WORKSPACE/.awf-home before planning. (src/microvm/workspace.ts does create it, but the CH backend does not use that module.)

So if /workspace/.awf-home is added to allowWrite without the host directory existing, planning fails closed with:

... not an existing path within a writable Cloud Hypervisor export: /workspace/.awf-home

The compiler must emit a mkdir -p "$GITHUB_WORKSPACE/.awf-home" step before the AWF invocation whenever it includes that path. This is documented as a prerequisite in docs/cloud-hypervisor-foundation.md and docs/awf-config-spec.md upstream — deliberately not auto-created, to preserve the existing-path/narrowing semantics.

Worth also confirming that /tmp/gh-aw/agent itself is created host-side before AWF starts; I could not find a compiler-generated mkdir for it, and the same fail-closed rule applies.


🟠 Blocker 3 — v0.28.5 is not a safe minimum for Cloud Hypervisor

AWFFilesystemAllowWriteMinVersion is v0.28.5, but selective allowWrite was broken on every real host until v0.28.6.

Firewall #7672 fixed a mount-propagation defect: a bind mount joins its source's peer group, so on any systemd host (including GitHub-hosted runners) every overlay arrived shared:N and the fail-closed propagation assertion aborted the run:

Staged mount tree propagation would leak: .../0-workspace/allowed has shared:1

This was caught only by live KVM CI; unit tests missed it because the test fake modelled mount --bind as producing an already-private mount.

Therefore: pinning >= v0.28.5 would advertise support for a version where CH allowWrite reliably fails. Please either raise AWFFilesystemAllowWriteMinVersion to v0.28.6, or introduce a separate CH-specific constant (e.g. AWFCloudHypervisorFilesystemAllowWriteMinVersion = "v0.28.6") so Docker/gVisor keep the lower bound while CH requires v0.28.6.


🟡 Stale schema description

pkg/workflow/schemas/awf-config.schema.json still carries the pre-v0.28.6 support statement:

"This option is currently supported by Docker and gVisor compose runtimes."

Released v0.28.6 says:

"This option is supported by the Docker and gVisor compose runtimes and by the Cloud Hypervisor microVM runtime; it is rejected with the sbx runtime and with Docker-in-Docker agent execution."

Re-sync the embedded schema from the released one, and grep for any other prose in docs/ asserting that allowWrite is unsupported under Cloud Hypervisor.


Proposed checklist

  • Decide the Cloud Hypervisor default allowWrite set and make ensureDefaultAgentWritePath runtime-aware (Blocker 1)
  • Emit mkdir -p for .awf-home (and confirm /tmp/gh-aw/agent) before the AWF invocation (Blocker 2)
  • Raise the allowWrite min version to v0.28.6, or add a CH-specific constant (Blocker 3)
  • Bump DefaultFirewallVersion v0.28.4v0.28.6
  • Re-sync pkg/workflow/schemas/awf-config.schema.json from released v0.28.6
  • Refresh embedded container digest pins
  • Regenerate pinned .lock.yml workflow artifacts
  • Add a .changeset/ patch entry (house style: "Bump the default gh-aw-firewall version to vX.Y.Z and refresh embedded container digest pins.")
  • Add compiler tests covering CH + allowWrite emission, the runtime-aware defaults, and the version gate boundaries

Suggested validation

Because the failure mode in Blocker 3 was invisible to unit tests, please validate at least one live Cloud Hypervisor run end-to-end rather than relying on generated-YAML assertions alone. Upstream, the equivalent coverage is the label-gated cloud-hypervisor-kvm job (~25 min).


Filed from the gh-aw-firewall side after landing #7669 / #7672 / #7670. Findings above were verified against the released v0.28.6 planner and the current gh-aw main. Happy to help review the compiler change.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions