diff --git a/.agents/skills/build-from-issue/SKILL.md b/.agents/skills/build-from-issue/SKILL.md index 8c843bfc6d..06d1324b02 100644 --- a/.agents/skills/build-from-issue/SKILL.md +++ b/.agents/skills/build-from-issue/SKILL.md @@ -1,6 +1,6 @@ --- name: build-from-issue -description: Given a GitHub issue number, plan and implement the work described in the issue. Operates iteratively - creates an implementation plan, responds to feedback, and only builds when the 'state:agent-ready' label is applied. Includes tests, documentation updates, and PR creation. Trigger keywords - build from issue, implement issue, work on issue, build issue, start issue. +description: Given a GitHub issue number, plan and implement the work described in the issue. Supports direct user requests and unattended queue processing through the `agent:*` workflow labels. Includes tests, documentation updates, and PR creation. Trigger keywords - build from issue, implement issue, work on issue, build issue, start issue. --- # Build From Issue @@ -14,16 +14,18 @@ This skill operates as a stateful workflow — it can be run repeatedly against - The `gh` CLI must be authenticated (`gh auth status`) - You must be in a git repository with a GitHub remote -## Critical: `state:agent-ready` Label Is Human-Only +## Invocation and Authorization -The `state:agent-ready` label is a **human gate**. It signals that a human has reviewed the plan and authorized the agent to build. Under **no circumstances** should this skill or any agent: +This skill supports two invocation modes: -- Apply the `state:agent-ready` label -- Ask the user to let the agent apply it -- Suggest automating its application -- Bypass the check by proceeding without it +- **Direct mode:** A user explicitly asks the agent to plan or implement a specific issue. The request itself authorizes the requested phase; the corresponding `agent:*` request label is not required. +- **Queue mode:** An always-on or unattended agent scans for work without a live user directing it to a specific issue. In this mode, `agent:plan-requested` authorizes planning and `agent:implementation-requested` authorizes implementation. -If the label is not present, the agent **must stop and wait**. This is a non-negotiable safety control — it ensures a human explicitly authorizes every build. +A direct request authorizes only what it says. A request to review or plan does not authorize implementation. A request to build, implement, or work on an issue authorizes both the planning needed to perform the work and implementation unless the user asks to stop after planning. + +The two request labels remain human-only queue controls. Under **no circumstances** should this skill or any agent apply them, ask to apply them, or suggest automating their application. + +Do not refuse a direct user request merely because its request label is absent. If direct work begins on an issue that was not already in the label-driven workflow, do not introduce `agent:in-progress` or `agent:pr-opened` solely for that invocation. If a matching request label is present, preserve the existing label transitions so unattended agents can track the workflow. ## Agent Comment Markers @@ -54,31 +56,43 @@ Each invocation follows this decision tree: ``` Fetch issue + comments │ - ├─ No plan comment (🏗️ build-plan) found? + ├─ topic:security present? + │ → Route to review-security-issue or fix-security-issue; STOP + │ + ├─ Triage incomplete, awaiting information, or awaiting human disposition? + │ → Report the blocking state and STOP + │ + ├─ state:accepted and roadmap association both absent? + │ → Human has not accepted the issue; STOP + │ + ├─ No plan comment and no direct planning request and agent:plan-requested absent? + │ → No request for agent planning; STOP + │ + ├─ No plan comment + direct planning request or agent:plan-requested present? │ → Generate plan via principal-engineer-reviewer │ → Post plan comment - │ → Add 'state:review-ready' label - │ → STOP + │ → Advance labels only for a label-driven invocation + │ → Continue if the direct request also authorized implementation; otherwise STOP │ ├─ Plan exists + new human comments since last agent response? │ → Respond to each comment (quote context, address feedback) │ → Update the plan comment if feedback requires plan changes │ → STOP │ - ├─ Plan exists + 'state:agent-ready' label + no 'state:in-progress' or 'state:pr-opened' label? + ├─ Plan exists + direct implementation request or 'agent:implementation-requested' label? │ → Run scope check (warn if high complexity) │ → Check for conflicting branches/PRs │ → BUILD (Steps 6–14) │ - ├─ 'state:in-progress' label present? + ├─ 'agent:in-progress' label present? │ → Detect existing branch and resume if possible │ → Otherwise report current state │ - ├─ 'state:pr-opened' label present? + ├─ 'agent:pr-opened' label present? │ → Report that PR already exists, link to it │ → STOP │ - └─ Plan exists + no new comments + no 'state:agent-ready'? + └─ Plan exists + no new comments + neither a direct implementation request nor 'agent:implementation-requested'? → Report: "Plan is posted and awaiting review. No new comments to address." → STOP ``` @@ -93,7 +107,15 @@ gh issue view --json number,title,body,state,labels,author If the issue is closed, report that and stop. -If the issue has the `state:triage-needed` label, report that the issue has not been triaged yet. Suggest using the `triage-issue` skill first to assess and classify the issue before planning implementation. Stop. +If `topic:security` is present, stop. General build agents must not plan or implement security issues. Route planning/review to `review-security-issue` and authorized remediation to `fix-security-issue`. + +Stop before planning in any of these states: + +- `state:triage-needed`: the issue has not been assessed; use `triage-issue`. +- `state:needs-info`: triage is waiting for evidence from the reporter. +- `state:validated` without roadmap placement: triage is complete, but a human has not yet decided whether OpenShell should invest in the work. + +Next, require a human acceptance signal: either `state:accepted` or placement on the roadmap. The label records acceptance without requiring scheduling; roadmap placement records acceptance and sequencing. If no plan exists, require either a direct user request for planning or the human-applied `agent:plan-requested` label before generating one. Never add or remove `state:accepted`, either human request label, or the `roadmap` label. ## Step 2: Fetch and Classify Comments @@ -117,7 +139,8 @@ Using the state machine above, determine what to do based on: 1. Whether a plan comment exists 2. Whether there are human comments newer than the last agent comment (plan or conversation) -3. Which labels are present (`state:review-ready`, `state:agent-ready`, `state:in-progress`, `state:pr-opened`) +3. Whether this is direct mode and which phase the user requested +4. Which disposition, roadmap, and agent-workflow labels are present (`state:accepted`, `agent:plan-requested`, `agent:plan-ready`, `agent:implementation-requested`, `agent:in-progress`, `agent:pr-opened`, and the `roadmap` label) Follow the appropriate branch below. @@ -125,7 +148,7 @@ Follow the appropriate branch below. ## Branch A: Generate the Plan -If no plan comment exists, generate one. +If no plan comment exists, generate one when the user directly requested planning or implementation, or when `agent:plan-requested` is present. Otherwise report that no one has requested agent planning and stop. ### A1: Analyze the Issue with Principal Engineer Reviewer @@ -137,7 +160,7 @@ Task tool with subagent_type="principal-engineer-reviewer" In the prompt, instruct the reviewer to: -1. Read the issue description thoroughly and identify what needs to change in the codebase. +1. Read the issue's user story and identify what needs to change in the codebase. Treat reporter diagnostics or solution ideas as optional context, not as authoritative or current analysis. 2. Map the requirements to existing code — read the relevant source files. 3. Determine the **issue type** — one of: `feat` (new feature), `fix` (bug fix), `refactor`, `chore`, `perf`, `docs`. 4. Propose the minimal set of changes that satisfies the requirements. @@ -151,6 +174,8 @@ In the prompt, instruct the reviewer to: 9. Assess **gateway config documentation impact** — if the change adds, removes, renames, or changes defaults for gateway TOML keys or driver-specific config options, the plan must include an update to `docs/reference/gateway-config.mdx`. If the change is surfaced through Helm or a compute-driver overview, also include `docs/reference/sandbox-compute-drivers.mdx` or the relevant deployment docs. 10. Assess **LSM compatibility** — if the change touches process identity, `/proc` filesystem access, binary execution, or inter-process visibility, flag whether it will behave differently on hosts running SELinux (enforcing) or AppArmor. In particular, tests that fork+exec into system binaries will fail on SELinux-enforcing hosts due to cross-label `/proc//exe` access restrictions. +Perform this investigation against the current branch and current product behavior. If the issue contains earlier diagnostics, verify them rather than relying on them. + ### A2: Post the Plan Comment Post the plan as a comment on the issue. This is the **canonical plan comment** that will be edited in place as the plan evolves. @@ -195,13 +220,15 @@ EOF )" ``` -### A3: Add the `state:review-ready` Label +### A3: Mark the Plan Ready in Queue Mode + +If `agent:plan-requested` was present, replace it with `agent:plan-ready`. Do not add `agent:plan-ready` for a direct invocation that was not already using the label workflow. ```bash -gh issue edit --add-label "state:review-ready" +gh issue edit --remove-label "agent:plan-requested" --add-label "agent:plan-ready" ``` -Report to the user that the plan has been posted and is awaiting review. Stop. +If the direct request authorized implementation, continue to Branch C. Otherwise report that the plan has been posted and stop. In queue mode, a human reviews the plan and applies `agent:implementation-requested` before an unattended agent can build. --- @@ -269,7 +296,7 @@ Report to the user what feedback was addressed and whether the plan was updated. ## Branch C: Build -If the plan exists and the `state:agent-ready` label is present (and neither `state:in-progress` nor `state:pr-opened` is set), proceed with implementation. +Proceed with implementation when the plan exists and either the user directly requested implementation or `agent:implementation-requested` is present. An existing `agent:in-progress` or `agent:pr-opened` label still triggers the resume or existing-PR checks below. ### Step 4: Scope Check @@ -279,7 +306,7 @@ Read the plan comment and check the **Complexity** and **Confidence** fields. > "This issue is rated High complexity / Low confidence. The plan includes open questions that may need human decisions during implementation. Proceeding, but flagging this for your awareness." - Continue — do not hard-stop. The human chose to apply `state:agent-ready`. + Continue — do not hard-stop. The user directly requested implementation or chose to apply `agent:implementation-requested`. ### Step 5: Conflict Detection @@ -324,10 +351,12 @@ git pull origin main git checkout -b -/$USERNAME ``` -### Step 7: Add `state:in-progress` Label +### Step 7: Mark Queue Work In Progress + +If `agent:implementation-requested` is present, replace it and `agent:plan-ready` with `agent:in-progress`. In direct mode without a request label, do not add an agent-workflow label. ```bash -gh issue edit --add-label "state:in-progress" +gh issue edit --remove-label "agent:implementation-requested" --remove-label "agent:plan-ready" --add-label "agent:in-progress" ``` ### Step 8: Implement the Changes @@ -377,7 +406,7 @@ Verification has two phases: unit tests + pre-commit, then E2E tests (if applica On each attempt: ```bash -# Run pre-commit checks (includes unit tests, linting, formatting) +# Run pre-commit checks (linting, formatting, license headers) mise run pre-commit ``` @@ -444,6 +473,8 @@ same branch. If the change affects user-facing compute-driver setup, also update `docs/reference/sandbox-compute-drivers.mdx` or the relevant deployment page. +Use the `sync-agent-infra` skill's maintenance map to identify related skill updates when the implementation changes behavior, commands, or development workflows. Run its full consistency check when the implementation adds, removes, or renames skills or crates; changes workflow relationships or skill coverage; modifies issue or PR templates; or changes agent cross-references. Fix any drift before committing. + ### Step 12: Commit and Push Commit all changes using conventional commit format. The `` comes from the issue type in the plan: @@ -592,10 +623,10 @@ Include **every test** that ran (not just the new ones) so the reviewer can see #### Update labels -Remove `state:in-progress` and `state:review-ready`, add `state:pr-opened`: +If `agent:in-progress` is present, replace it with `agent:pr-opened`. Do not add `agent:pr-opened` for an unlabeled direct invocation: ```bash -gh issue edit --remove-label "state:in-progress" --remove-label "state:review-ready" --add-label "state:pr-opened" +gh issue edit --remove-label "agent:in-progress" --add-label "agent:pr-opened" ``` #### Report workflow run URL @@ -613,7 +644,7 @@ Report the workflow run URL and suggest the user can use the `watch-github-actio ## Branch D: Resume In-Progress Build -If the `state:in-progress` label is present, the skill was previously started but may not have completed. +If the `agent:in-progress` label is present, the skill was previously started but may not have completed. 1. Check for an existing branch matching the issue ID: ```bash @@ -621,7 +652,7 @@ If the `state:in-progress` label is present, the skill was previously started bu ``` 2. If found, check it out and inspect the state (are there uncommitted changes? committed but not pushed? pushed but no PR?). 3. Resume from the appropriate step (9, 10, 12, or 13). -4. If the state is unrecoverable, report to the user and suggest starting fresh (remove `state:in-progress` label and re-run). +4. If the state is unrecoverable, report to the user and suggest starting fresh. Queue mode requires a human to reapply `agent:implementation-requested`; a new direct implementation request can resume without it. --- @@ -638,7 +669,7 @@ If the `state:in-progress` label is present, the skill was previously started bu | `gh pr list --state open --search "..."` | Search for open PRs | | `gh pr create --title "..." --body "..."` | Create a pull request | | `gh api user --jq '.login'` | Get current GitHub username | -| `mise run pre-commit` | Run pre-commit checks (includes unit tests, lint, format) | +| `mise run pre-commit` | Run pre-commit checks (lint, format, license headers) | | `mise run e2e:docker` | Run smoke E2E against a standalone Docker-backed gateway | | `mise run e2e:podman` | Run smoke E2E against a Podman-backed gateway | | `mise run e2e:vm` | Run smoke E2E against the VM compute driver | @@ -647,15 +678,16 @@ If the `state:in-progress` label is present, the skill was previously started bu ### First run — no plan exists -User says: "Build from issue #42" +User says: "Plan issue #42" 1. Fetch issue #42 — title: "Add pagination to dataset list endpoint" -2. Fetch comments — no `🏗️ build-plan` marker found -3. Pass issue to `principal-engineer-reviewer` for analysis -4. Reviewer produces a plan: feat type, Medium complexity, 3 implementation steps, unit + integration tests needed -5. Post the plan comment with the `🏗️ build-plan` marker -6. Add `state:review-ready` label -7. Report to user: "Plan posted on issue #42. Awaiting review." +2. Confirm `state:accepted` with no blocking triage state; the user's direct request authorizes planning even if `agent:plan-requested` is absent +3. Fetch comments — no `🏗️ build-plan` marker found +4. Pass issue to `principal-engineer-reviewer` for analysis +5. Reviewer produces a plan: feat type, Medium complexity, 3 implementation steps, unit + integration tests needed +6. Post the plan comment with the `🏗️ build-plan` marker +7. Because this direct invocation was unlabeled, leave the `agent:*` workflow labels unchanged +8. Report to user: "Plan posted on issue #42. Awaiting review." ### Second run — human left feedback @@ -677,29 +709,29 @@ User says: "Check issue #42" 4. Edit the plan comment to include search endpoint pagination — Revision 2 5. Report to user: "Updated plan to include search pagination (Revision 2)." -### Fourth run — state:agent-ready applied +### Fourth run — implementation requested User says: "Build issue #42" -1. Fetch issue #42 — labels include `state:agent-ready` +1. Fetch issue #42 — `state:accepted` is present; the user's direct request authorizes implementation 2. Plan exists (Revision 2), complexity: Medium, confidence: High 3. No conflicting branches or PRs 4. Create branch `feat/42-add-pagination/jmyers` -5. Add `state:in-progress` label +5. Leave `agent:*` labels unchanged because this direct invocation was not picked up from the queue 6. Implement pagination for both endpoints per the plan 7. Add unit tests for pagination logic, integration tests for both endpoints 8. `mise run pre-commit` passes on first attempt 9. E2E tests skipped (no changes under `e2e/`) 10. Commit, push, create PR with `Closes #42` 11. Post summary comment on issue with PR link -12. Update labels: remove `state:in-progress` + `state:review-ready`, add `state:pr-opened` +12. No agent-workflow label transition is needed 13. Report PR URL and workflow run status to user ### Run on issue with existing PR User says: "Build issue #42" -1. Fetch issue #42 — `state:pr-opened` label present +1. Fetch issue #42 — `agent:pr-opened` label present 2. Find existing PR #789 linked to the issue 3. Report: "PR [#789](...) already exists for issue #42. Nothing to build." @@ -707,7 +739,7 @@ User says: "Build issue #42" User says: "Build issue #99" -1. Fetch issue #99 — `state:agent-ready` label present +1. Fetch issue #99 — `state:accepted` is present; the user's direct request authorizes implementation 2. Plan exists: complexity High, confidence Low, has open questions 3. Warn user: "Issue #99 is rated High complexity / Low confidence. Proceeding but flagging for your awareness." 4. Continue with build diff --git a/.agents/skills/build-openshell-mxc-windows/SKILL.md b/.agents/skills/build-openshell-mxc-windows/SKILL.md new file mode 100644 index 0000000000..d61fc555e9 --- /dev/null +++ b/.agents/skills/build-openshell-mxc-windows/SKILL.md @@ -0,0 +1,341 @@ +--- +name: build-openshell-mxc-windows +description: Maintain and validate OpenShell's build-only Windows MSVC lane for x64 and ARM64. Use when working on Windows compilation, `windows:*` mise tasks, unsupported Windows compute-driver contracts, or Windows build reports. This skill does not implement Docker, Kubernetes, Podman, VM, MXC driver, policy translation, MSI, service, or supervisor runtime support on Windows. +--- + +# Build OpenShell-MXC for Windows + +This skill maintains the existing native Windows MSVC build lane in the +OpenShell repository. The Windows lane is already present in `main`; do not +treat this skill as a first-time porting recipe unless the user explicitly asks +for a new fork or a from-scratch bring-up. + +The lane is build-only. It validates that OpenShell can compile and test on +Windows MSVC for the supported deliverables: + +- `openshell-gateway.exe` +- `openshell.exe` + +It intentionally does not make Windows a Docker, Kubernetes, Podman, or VM +runtime host. + +## Current Repository Shape + +The Windows build lane is implemented by these tracked files: + +| Path | Purpose | +|---|---| +| `tasks/windows.toml` | Mise task entry points for `windows:*` commands. | +| `tasks/rust.toml`, `tasks/test.toml`, and `tasks/markdown.toml` | Windows routing for compiler-bearing checks, explicit Unix-only test skips, and Markdown dependency setup. | +| `tasks/scripts/windows-msvc.ps1` | PowerShell wrapper that enters the Visual Studio developer environment and invokes Cargo. | +| `.github/workflows/windows-msvc.yml` | Manually dispatched GitHub Actions jobs with architecture-specific Rust caches for x64 and future ARM64 Windows validation. | +| `architecture/windows-msvc-build.md` | Design notes and validation contract. | +| `.agents/skills/build-openshell-mxc-windows/` | This skill and companion reference material. | + +Use the code that is already in the repo. Do not generate a parallel Windows +build system, duplicate the wrapper, or add repository automation that the user +did not request. + +## Scope + +In scope: + +- Refreshing a local checkout to the latest upstream GitHub `main`. +- Maintaining `tasks/windows.toml` and `tasks/scripts/windows-msvc.ps1`. +- Running x64 and ARM64 MSVC checks. +- Building x64 and ARM64 release binaries for `openshell-gateway` and + `openshell`. +- Running workspace tests on a native x64 or ARM64 host. +- Running focused unsupported-driver contract tests. +- Reporting test counts, skipped/gated areas, warnings, artifacts, and logs. +- Keeping Linux and macOS build paths unchanged. +- Keeping unsupported Windows compute drivers explicit and testable. + +Out of scope: + +- Docker Desktop support on Windows. +- Kubernetes support on Windows. +- Podman, Podman machine, or Podman Desktop support on Windows. +- VM, Hyper-V, WSL, libkrun, or VM-backed sandbox execution on Windows. +- New MXC compute driver crate. +- OpenShell to MXC policy translation. +- Windows named-pipe driver IPC. +- Windows Credential Manager or DPAPI integration. +- MSI, WinGet, Windows service registration, or installer work. +- Windows supervisor runtime port. + +## Hard Rules + +- Do not enable Docker, Kubernetes, Podman, or VM runtimes on Windows. +- Do not build, package, ship, or smoke-test standalone Windows binaries for + unsupported compute drivers. +- Exclude unsupported Windows runtime crates from the Windows gateway dependency graph. +- Unsupported Windows runtime entry points must return a clear unsupported + error. +- Keep Windows-specific code behind `#[cfg(target_os = "windows")]`. +- Keep Unix/Linux-only code behind `#[cfg(unix)]` or + `#[cfg(target_os = "linux")]`. +- Do not modify the default Linux `mise run ci` path unless the user explicitly + asks for it. +- Use `mise run --skip-tools windows:*` for Windows validation. The Windows + toolchain is rustup plus Visual Studio Build Tools, not mise-provisioned Rust. +- Prefer one cross-platform `run` command when the underlying tool supports it + (for example, `npm --prefix`). Add `run_windows` only when the Windows shell + or validation contract genuinely differs. + +## Recommended Checkout Flow + +From a fork checkout where `upstream` points to the official +`NVIDIA/OpenShell` GitHub repository, use: + +```powershell +git fetch upstream main +git switch main +git merge --ff-only upstream/main +git branch --set-upstream-to=upstream/main main +git status --short --branch +``` + +For a direct checkout of the official repository, use `origin` instead of +`upstream`. Confirm the remote URLs with `git remote -v` before refreshing. + +If there are local changes, preserve or resolve them before refreshing. Do not +discard user work unless the user explicitly asks to clean the checkout. + +## Prerequisites + +The lane targets a Windows host with Visual Studio Build Tools and rustup. + +| Requirement | Check | Notes | +|---|---|---| +| Windows 11 | `[System.Environment]::OSVersion.Version` | Build 26100+ is recommended for MXC-adjacent validation, but compilation can still surface useful errors on older hosts. | +| Visual Studio 2022 or newer | `where.exe cl.exe` from a Developer PowerShell | Build Tools, Community, Professional, and Enterprise editions work when the target C++ components are installed. The wrapper discovers `VsDevCmd.bat` through `OPENSHELL_VSDEVCMD`, `vswhere`, or installed release directories such as `18` and `2022`. | +| Visual C++ ARM64 tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.ARM64 -property installationPath` | Required for native ARM64 check, build, and tests and for x64-to-ARM64 check/build. Tests always require a native runner. | +| Visual C++ ARM64 Spectre-mitigated libraries | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Runtimes.ARM64.Spectre -property installationPath` | Required by `regorus` through `msvc_spectre_libs`; the build fails when the selected MSVC toolset lacks `lib\spectre\arm64`. | +| Visual C++ Clang tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Llvm.Clang -property installationPath` | Provides host-native `libclang.dll` for `bindgen` and `clang-cl.exe` for ARM64 crypto dependencies such as `ring` and `aws-lc-sys`. On ARM64, the wrapper uses `VC\Tools\Llvm\Arm64\bin`. | +| Visual C++ CMake tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.CMake.Project -property installationPath` | Provides CMake and Ninja. The x64-to-ARM64 path adds Ninja to `PATH` for native dependencies but keeps bundled Z3 on CMake's Visual Studio ARM64 generator with native MSVC `cl.exe`. | +| Windows SDK | `where.exe rc.exe` from a Developer PowerShell | Install an SDK containing target libraries and ARM64 tools. | +| Rust via rustup | `rustc --version` | Add each target being validated: `x86_64-pc-windows-msvc` and/or `aarch64-pc-windows-msvc`. The wrapper also adds the selected target. | +| mise | `mise --version` | Used as a task runner only. | +| Git | `git --version` | Needed for checkout and sync work. | +| PowerShell | `$PSVersionTable.PSVersion` | Windows PowerShell 5.1 works; PowerShell 7 is quieter with mise shell hooks. | + +Do not install Visual Studio, Rust, Docker, Kubernetes, Podman, WSL, or Hyper-V +from this skill. + +## Environment Variables + +| Variable | Default | Purpose | +|---|---|---| +| `OPENSHELL_VSDEVCMD` | unset | Optional explicit path to `VsDevCmd.bat`. | +| `OPENSHELL_MXC_SKIP_ARM64` | `0` | Set to `1` to skip ARM64 when using `all` tasks. | +| `OPENSHELL_WINDOWS_BUILD_JOBS` | `CARGO_BUILD_JOBS`, then `4` | Positive Cargo job limit used by the wrapper. | +| `CARGO_TARGET_DIR` | `target` under repo root | Override Cargo output location. Use a short absolute path when x64-to-ARM64 builds approach Windows path-length limits. | +| `Z3_LIBRARY_PATH_OVERRIDE` | unset | Directory containing an x64 system `libz3.lib`; not valid for ARM64. | +| `Z3_SYS_Z3_HEADER` | unset | Full `z3.h` path required with a system Z3 library. | +| `Z3_SYS_BUNDLED_DIR_OVERRIDE` | pinned source cached under `CARGO_TARGET_DIR` when explicit, otherwise `%LOCALAPPDATA%\OpenShell\cache\z3` | Use an existing Z3 source tree containing `src/api/z3.h`; otherwise the wrapper fetches the pinned revision through Git and sets this automatically. | +| `RUSTC_WRAPPER` | cleared by wrapper | The wrapper clears inherited values because `--skip-tools` does not provision `sccache`. | + +Legacy fork variables such as `OPENSHELL_UPSTREAM`, +`OPENSHELL_MXC_FORK_DIR`, and `OPENSHELL_MXC_FORK_BRANCH` are no longer part +of the normal maintenance workflow. Use them only if the user explicitly asks +for a new disposable fork. + +## Validation Workflow + +Run the smallest useful slice first, then broaden: + +```powershell +mise run --skip-tools windows:check:x64 +mise run --skip-tools windows:check:arm64 +mise run --skip-tools windows:build:x64 +mise run --skip-tools windows:build:arm64 +mise run --skip-tools windows:test:x64 +mise run --skip-tools windows:test:unsupported:x64 +``` + +For full validation, detect the Windows host architecture first and choose the +native lane dynamically: + +```powershell +$arch = [System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture +switch ($arch.ToString()) { + "X64" { + mise run --skip-tools windows:ci + } + "Arm64" { + mise run --skip-tools windows:check:arm64 + mise run --skip-tools windows:build:arm64 + mise run --skip-tools windows:test:arm64 + mise run --skip-tools windows:test:unsupported:arm64 + mise run --skip-tools windows:artifacts + } + default { + throw "Unsupported Windows host architecture for OpenShell MSVC validation: $arch" + } +} +``` + +On x64 hosts, `windows:ci` is the full current CI contract and runs in this +order: + +1. x64 check. +2. ARM64 check, unless `OPENSHELL_MXC_SKIP_ARM64=1`. +3. x64 release build. +4. ARM64 release build, unless skipped. +5. Native x64 workspace tests. +6. Focused unsupported-driver contract tests. +7. Artifact reporting. + +The GitHub Actions jobs use architecture-specific `Swatinem/rust-cache` +entries for the Cargo registry and dependency target artifacts. Failed runs +also save their usable dependency artifacts. The workflow remains manually +dispatched until cache-hit runtimes justify restoring automatic triggers. + +The ARM64 check/build steps in this x64-host contract are cross-builds. The +wrapper discovers and adds host-native LLVM and Ninja to `PATH`, requires the +ARM64 compiler and Spectre-mitigated libraries, lets ARM64 crypto crates select +`clang-cl`, and keeps bundled Z3 on native MSVC `cl.exe` with CMake's Visual +Studio ARM64 generator. Z3 does not use Ninja because `z3-sys 0.10.9` passes +the MSBuild-only `-m` argument. + +On ARM64 hosts, validate the native ARM64 check, build, and test path. The +wrapper rejects test targets that do not match the host architecture, so x64 +compatibility under emulation is not part of these tasks. The aggregate +`windows:ci` task remains the x64-host CI contract; run the explicit ARM64 +commands above on an ARM64 host. + +The repository-wide `mise run pre-commit` task is also supported on Windows. +Its Rust check, Clippy, and test dependencies enter the same MSVC environment +for the native host target and clear inherited `RUSTC_WRAPPER`. Linux glibc +installer tests and Linux service/RPM packaging-asset tests skip explicitly; +the Linux build-environment shell-helper test also skips; cross-platform checks +continue to run. The blocking Windows Clippy pass excludes unsupported +Windows runtime packages as top-level targets. It allows only unused imports, +dead code, and unused async functions that result from cfg-gated Windows stubs; +other warnings remain errors. + +The wrapper limits Cargo to four jobs by default and serializes wrapper-owned +Cargo commands with a host-local mutex. It deliberately does not set `CL` or +`_CL_`: those variables are also consumed by `clang-cl`, where a global MSVC +option such as `/MP4` can be interpreted as an input file and break ARM64 +crypto dependency builds. + +## Expected Task Behavior + +| Task | Expected behavior | +|---|---| +| `windows:check:x64` | `cargo check --workspace` for `x86_64-pc-windows-msvc`, excluding unsupported Windows packages as top-level workspace targets. | +| `windows:check:arm64` | `cargo check --workspace` for `aarch64-pc-windows-msvc`, with the same top-level exclusions. | +| `windows:build:x64` | Release-builds `openshell-gateway.exe` and `openshell.exe` for x64. | +| `windows:build:arm64` | Release-builds `openshell-gateway.exe` and `openshell.exe` for ARM64. | +| `windows:test:x64` | Runs native x64 workspace tests with `--no-fail-fast`, excluding unsupported Windows packages as top-level workspace targets. | +| `windows:test:arm64` | Runs native ARM64 workspace tests with `--no-fail-fast` and the same package exclusions. Rejects non-ARM64 hosts. | +| `windows:test:unsupported:x64` | Re-runs focused `openshell-server` tests for unsupported Windows driver behavior. | +| `windows:test:unsupported:arm64` | Re-runs the same focused contracts natively on ARM64. Rejects non-ARM64 hosts. | +| `windows:artifacts` | Reports size and SHA256 for release artifacts that exist. | +| `windows:ci` | Runs the full ordered x64-host Windows CI lane, plus ARM64 check/build when not skipped. | + +The unsupported driver package excludes are intentional. They prevent standalone +driver crates from being top-level Windows check/test targets while allowing +required libraries and Windows contracts to compile through gateway dependencies. +This includes the Kubernetes Secrets and Vault packages: their libraries remain +in the gateway build graph, but their Unix-socket standalone binaries do not. + +## Unsupported Driver Contract + +Windows must continue to reject unsupported compute drivers clearly. + +| Driver | Windows build behavior | Runtime behavior | +|---|---|---| +| Docker | Driver crate excluded; server config contract retained. | Gateway construction returns unsupported. | +| Kubernetes | Driver crate excluded; server config contract retained. | Gateway construction returns unsupported. | +| Podman | Driver crate excluded; server config contract retained. | Gateway construction returns unsupported. | +| VM | Driver crate excluded from workspace validation. | Gateway construction returns unsupported. | + +The focused contract tasks for either native architecture run: + +```text +windows_builtin_compute_drivers_report_unsupported +``` + +These tests are also included in the full x64 workspace test run; the focused +task intentionally re-runs them so unsupported Windows behavior is visible in +the CI report. + +## Test Accounting Guidance + +When reporting `windows:ci`, distinguish these categories: + +- Passed tests from the full x64 workspace test log. +- Passed tests from the full ARM64 workspace test log when run on a native + ARM64 host. +- The focused unsupported-contract re-run. +- Explicit Cargo ignored tests, usually ignored doc examples. +- Tests hidden by `#[cfg(not(target_os = "windows"))]`; these often appear as + `running 0 tests`, not as ignored tests. +- Test-name `filtered out` counts from focused `cargo test` invocations. +- Package-level exclusions for unsupported Windows crates; Cargo does not report + those as ignored tests. + +Useful log files: + +| Log | Meaning | +|---|---| +| `build-x86_64-pc-windows-msvc-check.log` | x64 check output. | +| `build-aarch64-pc-windows-msvc-check.log` | ARM64 check output. | +| `build-x86_64-pc-windows-msvc-release.log` | x64 release build output. | +| `build-aarch64-pc-windows-msvc-release.log` | ARM64 release build output. | +| `test-x86_64-pc-windows-msvc.log` | Full native x64 workspace test output. | +| `test-aarch64-pc-windows-msvc.log` | Full native ARM64 workspace test output. | +| `test-x86_64-pc-windows-msvc-unsupported-*.log` | Focused unsupported-driver contract output. | +| `test-aarch64-pc-windows-msvc-unsupported-*.log` | Focused native ARM64 contract output. | + +The first bundled-Z3 check or test can spend several minutes in CMake/MSBuild +without much console output because Cargo output is redirected to the log. Look +for native `MSBuild.exe` workers before treating the process as stalled. The +wrapper fetches the pinned Z3 source through Git before Cargo starts. It caches +under an explicitly configured `CARGO_TARGET_DIR`, or under the current user's +local application data directory when Cargo uses its default target tree. +Concurrent commands publish the validated source through an atomic directory +rename, so x64 and ARM64 validation can share the cache safely. The wrapper does +not rely on the rate-limited GitHub Contents API used by `z3-sys`. A failed +fetch reports the partial checkout path for diagnosis. The artifact report +computes SHA256 through .NET directly and does not rely on the +`Get-FileHash` module being available inside the mise-launched Windows +PowerShell process. + +## Common Fix Patterns + +When Windows validation fails: + +1. Identify whether the error is from a top-level Windows deliverable, a + gateway dependency stub, or a Unix-only module leaking into the Windows build. +2. Prefer existing local patterns in the same crate. +3. Gate Unix imports and modules with `#[cfg(unix)]` or + `#[cfg(target_os = "linux")]`. +4. Add or preserve Windows stubs that return unsupported errors. +5. Keep Linux behavior unchanged. +6. Run `cargo fmt --all`, `git diff --check`, and the relevant `windows:*` + tasks after changes. + +Do not add broad abstractions or new Windows runtime support to satisfy a build +error. If a missing runtime feature is required, stop and propose a follow-on +skill or design doc. + +## Final Report Checklist + +Every substantial Windows build run should report: + +| Item | Required detail | +|---|---| +| Git state | Branch, upstream GitHub base commit, and whether local changes existed. | +| Host preconditions | OS, Rust, MSVC discovery, and notable warnings. | +| Commands run | Exact `mise run --skip-tools windows:*` commands. | +| x64 check/build | Pass/fail and log path. | +| ARM64 check/build | Pass/fail/skipped and log path. | +| Native tests | Passed/failed/ignored/filtered counts and log path for the host architecture. | +| Unsupported contracts | Which focused tests ran and their result. | +| Artifacts | Binary paths, size, and SHA256 when available. | +| Skips | Explicitly explain tests not run for a non-native architecture, unsupported driver package exclusions, and Windows cfg-gated tests. | +| Follow-ups | Only concrete follow-ups tied to failures or requested scope. | diff --git a/.agents/skills/build-openshell-mxc-windows/reference.md b/.agents/skills/build-openshell-mxc-windows/reference.md new file mode 100644 index 0000000000..4a5f2668a1 --- /dev/null +++ b/.agents/skills/build-openshell-mxc-windows/reference.md @@ -0,0 +1,230 @@ +# Reference: Windows MSVC maintenance lane + +Companion to [SKILL.md](SKILL.md). Use this file for quick lookup while +maintaining the existing build-only Windows MSVC lane. + +## Lane Files + +| File | Purpose | +|---|---| +| `tasks/windows.toml` | Mise task definitions for `windows:*`. | +| `tasks/scripts/windows-msvc.ps1` | Visual Studio environment discovery, rustup target setup, Cargo invocation, logs, artifact report. | +| `.github/workflows/windows-msvc.yml` | Manual GitHub Actions x64 job and disabled ARM64 scaffold, each with an architecture-specific Rust dependency cache. | +| `architecture/windows-msvc-build.md` | Human-readable design contract. | + +## Commands + +Use `--skip-tools` for all Windows mise tasks: + +```powershell +mise run --skip-tools windows:check:x64 +mise run --skip-tools windows:check:arm64 +mise run --skip-tools windows:build:x64 +mise run --skip-tools windows:build:arm64 +mise run --skip-tools windows:test:x64 +mise run --skip-tools windows:test:arm64 +mise run --skip-tools windows:test:unsupported:x64 +mise run --skip-tools windows:test:unsupported:arm64 +mise run --skip-tools windows:ci +``` + +For host-native full validation, detect architecture first: + +```powershell +$arch = [System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture +if ($arch -eq [System.Runtime.InteropServices.Architecture]::Arm64) { + mise run --skip-tools windows:check:arm64 + mise run --skip-tools windows:build:arm64 + mise run --skip-tools windows:test:arm64 + mise run --skip-tools windows:test:unsupported:arm64 + mise run --skip-tools windows:artifacts +} else { + mise run --skip-tools windows:ci +} +``` + +The native test tasks reject a target that does not match the host architecture. +Do not report x64 compatibility-under-emulation coverage from an ARM64 run. + +The wrapper adds missing rustup targets and clears inherited +`RUSTC_WRAPPER`. It does not install Visual Studio, Rust, Docker, Kubernetes, +Podman, WSL, Hyper-V, or VM tooling. + +On Windows, `mise run pre-commit` routes `rust:check`, `rust:lint`, and +`test:rust` through this wrapper for the host-native target. The shared task +definitions retain their existing Unix commands. Only tests for Linux glibc +installer behavior, Linux build-environment shell helpers, and Linux +service/RPM packaging assets skip on Windows. The Windows Clippy command +excludes unsupported runtime packages as top-level targets and allows only +unused imports, dead code, and unused async functions caused by cfg-gated +Windows stubs; other warnings remain errors. + +The wrapper limits Cargo to four jobs by default and serializes wrapper-owned +Cargo commands with a host-local mutex. It does not set `CL` or `_CL_` because +`clang-cl` also consumes them and can parse a global `/MP4` option as an input +file. + +For ARM64, verify the Visual Studio instance contains the ARM64 MSVC tools, +ARM64 Spectre-mitigated libraries, Clang tools, CMake tools, and a Windows SDK. +Clang supplies host-native `libclang.dll` for `bindgen` and `clang-cl.exe` for +ARM64 crypto dependencies such as `ring` and `aws-lc-sys`. Native ARM64 uses +the normal bundled-Z3 CMake path. An x64-to-ARM64 check/build discovers and +adds host-native Ninja to `PATH`, while the crypto crates select `clang-cl`. +Bundled Z3 uses CMake's Visual Studio ARM64 generator with native MSVC `cl.exe` +because `z3-sys 0.10.9` passes the MSBuild-only `-m` argument. Use a short +`CARGO_TARGET_DIR` if Windows path-length limits are reached. + +## Unsupported Driver Rules + +Windows is a build target only. These runtimes remain unsupported: + +- Docker +- Kubernetes +- Podman +- VM + +Rules: + +- Keep config/library stubs where the gateway needs them. +- Return clear unsupported errors at runtime. +- Do not build standalone Windows driver binaries. +- Do not add Docker Desktop, WSL, Hyper-V, Podman machine, Podman Desktop, or + VM-backed execution as part of this skill. + +Current focused unsupported-contract tests: + +```text +windows_builtin_compute_drivers_report_unsupported +``` + +Run them with the architecture-specific focused task on the native host. + +## Cargo Excludes + +The Windows wrapper intentionally excludes unsupported runtime packages as +top-level workspace targets for check/test: + +```text +--exclude openshell-driver-docker +--exclude openshell-driver-kubernetes +--exclude openshell-driver-kubernetes-secrets +--exclude openshell-driver-podman +--exclude openshell-driver-vault +--exclude openshell-driver-vm +--exclude openshell-sandbox +--exclude openshell-supervisor-network +--exclude openshell-supervisor-process +--exclude openshell-vfio +``` + +The gateway keeps platform configuration and unsupported-operation contracts +without depending on the Docker, Kubernetes, Podman, sandbox supervisor, +process supervisor, VM, or VFIO runtime crates. The Kubernetes Secrets and +Vault libraries still compile as gateway dependencies; only their standalone +Unix-socket binaries and package-level tests are excluded as top-level targets. + +## Common Errors + +### Unix imports leak into Windows builds + +Symptoms: + +```text +unresolved import std::os::unix +unresolved import tokio::net::UnixListener +unresolved import nix::... +``` + +Fix pattern: + +```rust +#[cfg(unix)] +use tokio::net::{UnixListener, UnixStream}; +``` + +Move Unix-only functions into Unix-only modules, or add a Windows stub that +returns an unsupported error. + +### Linux-only dependency reaches Windows + +Symptoms: + +```text +failed to run custom build command for libseccomp-sys +pkg-config could not find libsecret +``` + +Fix pattern: + +```toml +[target.'cfg(target_os = "linux")'.dependencies] +libseccomp = "..." +``` + +Only gate the dependency if no Windows path should use it. + +### ARM64 check fails but x64 passes + +Likely causes: + +- Native dependency does not support `aarch64-pc-windows-msvc`. +- ARM64 MSVC or Spectre-mitigated libraries are missing. +- Host-native `clang-cl`, Ninja, or CMake is missing during an x64-to-ARM64 build. +- `CL` or `_CL_` injects a global MSVC option such as `/MP4` into `clang-cl`. +- Build script assumes x64 tools. +- Inline assembly or prebuilt artifact lacks ARM64 handling. + +Do not skip ARM64 silently. Either fix the target handling or report the exact +blocked dependency. + +### Focused tests report many filtered-out tests + +This is expected for `windows:test:unsupported:x64`. Cargo runs one named test +and filters the other `openshell-server` tests. Report these as filtered, not +ignored. + +## Reporting Counts + +Use the log summaries from: + +| Log | Count source | +|---|---| +| `test-x86_64-pc-windows-msvc.log` | Full x64 workspace test pass. | +| `test-aarch64-pc-windows-msvc.log` | Full native ARM64 workspace test pass. | +| `test-x86_64-pc-windows-msvc-unsupported-*.log` | Focused unsupported-contract re-runs and filtered counts. | +| `test-aarch64-pc-windows-msvc-unsupported-*.log` | Focused native ARM64 re-runs and filtered counts. | + +Separate: + +- passed +- failed +- ignored +- filtered out +- cfg-gated zero-test targets +- package-level excludes + +Package-level excludes are not printed as ignored tests by Cargo. + +## Final Sanity Checks + +Before committing Windows-lane changes, choose checks based on the host +architecture: + +```powershell +cargo fmt --all +git diff --check +$arch = [System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture +if ($arch -eq [System.Runtime.InteropServices.Architecture]::Arm64) { + mise run --skip-tools windows:check:arm64 + mise run --skip-tools windows:build:arm64 + mise run --skip-tools windows:test:arm64 + mise run --skip-tools windows:test:unsupported:arm64 +} else { + mise run --skip-tools windows:check:x64 + mise run --skip-tools windows:check:arm64 + mise run --skip-tools windows:test:unsupported:x64 +} +``` + +Run the full x64-host `windows:ci` lane when build or test behavior changed and +the host can run that lane natively. diff --git a/.agents/skills/create-github-issue/SKILL.md b/.agents/skills/create-github-issue/SKILL.md index b1311a297e..609d55ad15 100644 --- a/.agents/skills/create-github-issue/SKILL.md +++ b/.agents/skills/create-github-issue/SKILL.md @@ -17,22 +17,27 @@ This project uses YAML form issue templates. When creating issues, match the tem ### Bug Reports -Do not add a type label automatically. The body must include an **Agent Diagnostic** section — this is required by the template and enforced by project convention. Apply area or topic labels only when they are clearly known. +Do not add a type label automatically. The body must include a **User Story**, **Problem Statement**, **Impact / Why This Matters**, and **Acceptance Criteria**, followed by bug-specific reproduction steps and environment details. Logs are optional and must be concise and redacted. Apply area or topic labels only when they are clearly known. ```bash gh issue create \ --title "bug: " \ --body "$(cat <<'EOF' -## Agent Diagnostic +## User Story - +As a , I want , so that . -## Description +## Problem Statement + + + +## Impact / Why This Matters -**Actual behavior:** + -**Expected behavior:** +## Acceptance Criteria + +- [ ] ## Reproduction Steps @@ -41,14 +46,14 @@ What was found? What was tried?> ## Environment -- OS: -- Docker: - OpenShell: +- OS: +- Runtime, deployment, or integration: ## Logs ``` - + ``` EOF )" @@ -56,28 +61,39 @@ EOF ### Feature Requests -Do not add a type label automatically. The body must include a **Proposed Design** — not a "please build this" request. Apply area or topic labels only when they are clearly known. +Do not add a type label automatically. The body must include a **User Story**, **Problem Statement**, **Impact / Why This Matters**, **Proposed Design**, **Acceptance Criteria**, and **Alternatives Considered**. The proposed design should define the user-facing workflow and externally observable behavior without prescribing internal implementation. Agent investigation is optional. Apply area or topic labels only when they are clearly known. ```bash gh issue create \ --title "feat: " \ --body "$(cat <<'EOF' +## User Story + +As a , I want , so that . + ## Problem Statement - + + +## Impact / Why This Matters + + ## Proposed Design - + + +## Acceptance Criteria + +- [ ] ## Alternatives Considered - + ## Agent Investigation - + EOF )" ``` @@ -107,6 +123,8 @@ EOF GitHub built-in issue types (`Bug`, `Feature`, `Task`) should come from the matching issue template when possible, or be set manually afterward. Do not try to emulate them through labels. +Creating an issue does not accept it or queue agent work. Agents never apply `state:accepted`, the `roadmap` label, add issues to the roadmap project, or apply `agent:plan-requested` or `agent:implementation-requested`. Community issues proceed through `triage-issue`; a human accepts technically validated work with `state:accepted` or roadmap placement. The request labels queue work for unattended agents; a user may instead direct an agent to a specific issue. + ## Useful Options | Option | Description | diff --git a/.agents/skills/create-github-pr/SKILL.md b/.agents/skills/create-github-pr/SKILL.md index 8690783ea4..d98aba37f2 100644 --- a/.agents/skills/create-github-pr/SKILL.md +++ b/.agents/skills/create-github-pr/SKILL.md @@ -11,7 +11,7 @@ Create pull requests on GitHub using the `gh` CLI. - The `gh` CLI must be authenticated (`gh auth status`) - You must have commits on a branch that's pushed to the remote -- Branch should follow naming convention: `-/` +- For issue-backed work, the branch should follow `-/`. Exempt issue-less changes may use `/`. ## Before Creating a PR @@ -24,6 +24,10 @@ in the same branch. If the change affects user-facing compute-driver setup, also update `docs/reference/sandbox-compute-drivers.mdx` or the relevant deployment docs. +### Check Agent Infrastructure + +Use the `sync-agent-infra` skill's maintenance map to identify related skill updates when the branch changes behavior, commands, or development workflows. Run its full consistency check when the branch adds, removes, or renames skills or crates; changes workflow relationships or skill coverage; modifies issue or PR templates; or changes agent cross-references. Resolve any drift before creating the PR. + ### Run Pre-commit Checks Run the local pre-commit task before opening a PR: @@ -43,7 +47,7 @@ Before creating a PR, verify: git branch --show-current ``` -2. **Branch follows naming convention** - Format: `-/` +2. **Branch follows naming convention** - Use `-/` for issue-backed work or `/` for an exempt issue-less change. ```bash # Example: 1234-add-pagination/jd @@ -110,7 +114,7 @@ gh pr create --title "PR title" --body "PR description" ### Link to an Issue -Use `Closes #` in the body to auto-close the issue when merged: +Features, user-visible behavior changes, public API changes, architecture changes, and multi-PR efforts must link an accepted issue. Use `Closes #` in the body to auto-close the issue when merged: ```bash gh pr create \ @@ -122,6 +126,8 @@ gh pr create \ - Returns 400 instead of 500" ``` +Small documentation fixes, mechanical maintenance, and obvious localized bug fixes may omit a separate issue when the PR contains enough context to review the decision and implementation together. In that case, write `No issue required: ` in the Related Issue section. Do not use this exception for security fixes; follow `SECURITY.md`. + ### Create as Draft For work-in-progress that's not ready for review: @@ -153,7 +159,7 @@ PR descriptions must follow the project's [PR template](.github/PULL_REQUEST_TEM ## Related Issue - + ## Changes diff --git a/.agents/skills/create-rfc/SKILL.md b/.agents/skills/create-rfc/SKILL.md new file mode 100644 index 0000000000..f767e47587 --- /dev/null +++ b/.agents/skills/create-rfc/SKILL.md @@ -0,0 +1,51 @@ +--- +name: create-rfc +description: Create OpenShell RFC proposals in rfc/ from a design request. Use when the user asks to write, draft, start, create, or update an RFC, Request for Comments, architecture proposal, API proposal, process proposal, or cross-cutting design proposal that should follow the OpenShell RFC process and template. +--- + +# Create RFC + +## Workflow + +Create RFCs by following `rfc/README.md` and `rfc/0000-template/README.md`. +Keep the template as the source of truth for section guidance. + +1. Read `rfc/README.md` to confirm when an RFC is appropriate, how to choose the + RFC number, and how the lifecycle works. +2. Read `rfc/0000-template/README.md` before drafting. Follow its section + guidance, including scope, expected detail, and suggested section length. +3. Choose the next available `NNNN` from the existing `rfc/NNNN-*` directories + unless the user provided a specific number. +4. Create `rfc/NNNN-short-title/README.md` by copying the template and replacing + placeholders. Use a short hyphenated folder title. +5. Fill in front matter with the RFC author, `state: draft`, and any related + links the user provided. If the author is unknown, use the requesting user's + GitHub handle when available or leave the template placeholder. +6. Draft each section from the user's design context. Keep Summary concise, + Motivation readable by anyone, Non-goals explicit, Proposal focused on what + is being proposed, and Alternatives focused on credible competing approaches. +7. Preserve uncertainty in Open questions instead of silently deciding unknowns. + If a missing decision blocks a coherent RFC, ask the user for that decision. +8. Check the completed RFC against the template once more before finishing. + +## Writing Standards + +- Prefer concrete design statements over placeholder language. +- Link to relevant issues, prior RFCs, and architecture docs when they provide + needed context. +- Keep rejected or left-out designs in Alternatives, not Proposal. +- Use Mermaid diagrams for architecture or data flow when a diagram would make + the proposal easier to review. +- Do not update `architecture/` or published docs just because an RFC was + drafted. Those updates belong with implementation or with an accepted RFC when + the user asks for them. + +## Validation + +Before handing the RFC back to the user: + +- Verify the folder name and RFC number match the process in `rfc/README.md`. +- Verify every template section is present or intentionally marked as not + applicable. +- Run a Markdown formatting or lint check only if the repo already provides one + for Markdown-only changes. diff --git a/.agents/skills/create-rfc/agents/openai.yaml b/.agents/skills/create-rfc/agents/openai.yaml new file mode 100644 index 0000000000..8717be4ff2 --- /dev/null +++ b/.agents/skills/create-rfc/agents/openai.yaml @@ -0,0 +1,4 @@ +interface: + display_name: "Create RFC" + short_description: "Create OpenShell RFC proposals" + default_prompt: "Use $create-rfc to draft an OpenShell RFC from this design." diff --git a/.agents/skills/create-spike/SKILL.md b/.agents/skills/create-spike/SKILL.md index 3c09d20de1..63bcffb72b 100644 --- a/.agents/skills/create-spike/SKILL.md +++ b/.agents/skills/create-spike/SKILL.md @@ -5,7 +5,7 @@ description: Investigate a plain-language problem description by deeply explorin # Create Spike -Investigate a problem, map it to the codebase, and produce a structured GitHub issue ready for `build-from-issue`. +Investigate a problem, map it to the codebase, and produce a structured GitHub issue ready for human disposition and roadmap placement. A **spike** is an exploratory investigation. The user has a vague idea — a feature they want, a bug they've noticed, a performance concern — but hasn't mapped it to code, assessed feasibility, or structured it as a buildable issue. This skill does that mapping. @@ -122,7 +122,9 @@ Based on the investigation results, select appropriate labels: - **Do not add issue type labels** — GitHub built-in issue types come from issue templates or manual follow-up, not labels - **Include area labels** if they exist in the repo (e.g., `area:sandbox`, `area:proxy`, `area:policy`, `area:cli`) - **Do not invent labels** — only use labels that already exist in the repo -- **Add `state:review-ready`** — the issue is ready for human review upon creation +- **Add `state:validated` only when the evidence is sufficient for human disposition** — the spike established a coherent problem or proposal and completed the factual assessment needed for a human yes/no decision +- **Add `state:needs-info` instead when material evidence is missing** — identify the exact evidence, reproduction details, or decision input still needed in the issue body +- **Never add `state:accepted`, an `agent:*` label, or the `roadmap` label** — acceptance, roadmap placement, and requests for agent work require a human decision ## Step 4: Create the GitHub Issue @@ -131,7 +133,7 @@ Create the issue with a structured body containing both the stakeholder-readable ```bash gh issue create \ --title ": " \ - --label "" --label "state:review-ready" \ + --label "" --label "" \ --body "$(cat <<'EOF' ## Problem Statement @@ -195,6 +197,12 @@ gh issue create \ - - ... +## Disposition Readiness + +- **State:** `` +- **Assessment:** +- **Missing evidence:** + ## Test Considerations - @@ -203,7 +211,7 @@ gh issue create \ - --- -*Created by spike investigation. Use `build-from-issue` to plan and implement.* +*Created by spike investigation. `state:validated` means the issue is ready for human disposition; `state:needs-info` means specific evidence is still required. A human applies `state:accepted` or places the issue on the roadmap if OpenShell should pursue the work. To queue unattended agent planning, a human applies `agent:plan-requested`; a direct request to an agent does not require that label.* EOF )" ``` @@ -225,7 +233,13 @@ After creating the issue, report: 3. Key risks or decisions that need human attention 4. Next steps: -> Review the issue. Refine the proposed approach if needed, then use `build-from-issue` on the issue to create an implementation plan and build it. +For `state:validated`: + +> Review the issue and decide whether OpenShell should pursue it. If yes, apply `state:accepted`, associate it with a roadmap item, or do both. Either action records acceptance; roadmap placement additionally records sequencing. The work may remain human-owned. Apply `agent:plan-requested` to queue planning for an unattended agent, or directly ask an agent to use `build-from-issue`. If no, close it as not planned and record the rationale. + +For `state:needs-info`: + +> Collect the missing evidence identified in the issue. Leave it off the roadmap. Once the evidence is sufficient, replace `state:needs-info` with `state:validated` for human disposition. ## Design Principles @@ -239,6 +253,8 @@ After creating the issue, report: 5. **Cross-reference `build-from-issue`.** Mention it as the natural next step in the issue body footer. +6. **Treat validation as an evidence threshold, not an automatic spike outcome.** Apply `state:validated` only when the investigation supports a human accept/decline decision. Otherwise apply `state:needs-info`, state what is missing, and leave the issue off the roadmap. + ## Useful Commands Reference | Command | Description | @@ -263,9 +279,9 @@ User says: "Allow sandbox egress to private IP space via networking policy" - Reads `architecture/security-policy.md` and `architecture/sandbox.md` - Identifies exact insertion points: policy field addition, SSRF check bypass path, OPA rule extension - Assesses: Medium complexity, High confidence, ~6 files -3. Fetch labels — select `area:sandbox`, `area:proxy`, `area:policy`, `state:review-ready` +3. Fetch labels — select `area:sandbox`, `area:proxy`, `area:policy`, `state:validated` 4. Create issue: `feat: allow sandbox egress to private IP space via networking policy` — body includes both the summary and full investigation (code references, architecture context, alternative approaches) -5. Report: "Created issue #59. The investigation found that private IP blocking is enforced at the SSRF check layer in the proxy. The proposed approach adds a policy-level override. Review the issue and use `build-from-issue` when ready." +5. Report: "Created issue #59. The investigation found that private IP blocking is enforced at the SSRF check layer in the proxy. The proposed approach adds a policy-level override. A human must now accept or decline it and place it on the roadmap if accepted." ### Bug investigation spike @@ -279,9 +295,9 @@ User says: "The proxy retry logic seems too aggressive — I'm seeing cascading - Maps the failure propagation path - Identifies that retries happen without backoff jitter, causing thundering herd - Assesses: Low complexity, High confidence, ~2 files -3. Fetch labels — select `area:proxy`, `state:review-ready` +3. Fetch labels — select `area:proxy`, `state:validated` 4. Create issue: `fix: proxy retry logic causes cascading failures under load` — body includes both the summary and full investigation (retry code references, current behavior trace, comparison to standard backoff patterns) -5. Report: "Created issue #74. The proxy retries without jitter or circuit breaking, which amplifies failures under load. Straightforward fix. Review and use `build-from-issue` when ready." +5. Report: "Created issue #74. The proxy retries without jitter or circuit breaking, which amplifies failures under load. A human must now accept or decline it and place it on the roadmap if accepted." ### Performance/refactoring spike @@ -295,6 +311,6 @@ User says: "Policy evaluation is getting slow — can we cache compiled OPA poli - Reads the policy reload/hot-swap mechanism - Identifies that policies are recompiled on every evaluation - Assesses: Medium complexity, Medium confidence (cache invalidation is a design decision), ~4 files -3. Fetch labels — select `area:policy`, `state:review-ready` +3. Fetch labels — select `area:policy`, `state:validated` 4. Create issue: `perf: cache compiled OPA policies to reduce evaluation latency` — body includes both the summary and full investigation (compilation hot path, per-request overhead, cache invalidation strategies with trade-offs) -5. Report: "Created issue #81. Policies are recompiled per-request with no caching. The main design decision is the cache invalidation strategy — flagged as an open question. Review and use `build-from-issue` when ready." +5. Report: "Created issue #81. Policies are recompiled per-request with no caching. The main design decision is the cache invalidation strategy. A human must now accept or decline it and place it on the roadmap if accepted." diff --git a/.agents/skills/debug-inference/SKILL.md b/.agents/skills/debug-inference/SKILL.md index 6770da5987..08462a4751 100644 --- a/.agents/skills/debug-inference/SKILL.md +++ b/.agents/skills/debug-inference/SKILL.md @@ -1,6 +1,6 @@ --- name: debug-inference -description: Debug why inference.local or external inference setup is failing. Use when the user cannot reach a local model server, has provider base URL issues, sees inference verification failures, hits protocol mismatches, or needs to diagnose inference on local vs remote gateways. Trigger keywords - debug inference, inference.local, local inference, ollama, vllm, sglang, trtllm, NIM, inference failing, model server unreachable, failed to verify inference endpoint, host.openshell.internal. +description: Debug why inference.local, direct external inference, or supervisor-only system inference is failing. Use when the user cannot reach a local model server, has provider base URL issues, sees inference verification failures, hits protocol mismatches, or needs to diagnose inference on local vs remote gateways. Trigger keywords - debug inference, inference.local, system inference, sandbox-system, local inference, ollama, vllm, sglang, trtllm, NIM, inference failing, model server unreachable, failed to verify inference endpoint, host.openshell.internal. --- # Debug Inference @@ -11,7 +11,7 @@ Use `openshell` CLI commands to inspect the active gateway, provider records, ma ## Overview -OpenShell supports two different inference paths. Diagnose the correct one first. +OpenShell supports three inference paths. Diagnose the correct one first. 1. **Managed inference** through `https://inference.local` - Configured by `openshell inference set` @@ -21,6 +21,10 @@ OpenShell supports two different inference paths. Diagnose the correct one first - Controlled by `network_policies` - Requires the application to call the external host directly - Requires provider attachment and network access to be configured separately +3. **System inference** used by platform functions + - Configured by `openshell inference set --system` + - Uses the `sandbox-system` route + - Consumed in-process by the sandbox supervisor and not exposed to sandbox user code through `inference.local` For local or self-hosted engines such as Ollama, vLLM, SGLang, TRT-LLM, and many NIM deployments, the most common managed inference pattern is an `openai` provider with `OPENAI_BASE_URL` pointing at a host the gateway can reach. @@ -38,10 +42,13 @@ Use these commands first: # Which gateway is active, and can the CLI reach it? openshell status -# Show managed inference config for inference.local +# Show both the user-facing and system inference routes openshell inference get -# Inspect the provider record referenced by inference.local +# Show only the supervisor-only system route +openshell inference get --system + +# Inspect the provider record referenced by the relevant route openshell provider get # Inspect gateway topology details when remote/local confusion is suspected @@ -59,9 +66,9 @@ When the user asks to debug inference, run diagnostics automatically in this ord Establish these facts first: -1. Is the application calling `https://inference.local` or a direct external host? +1. Is sandbox code calling `https://inference.local`, is the application calling a direct external host, or is a platform function using system inference? 2. Which gateway is active, and is it local, remote, or cloud? -3. Which provider and model are configured for managed inference? +3. Which provider, model, and timeout are configured for the relevant route? 4. Is the upstream local to the gateway host, or somewhere else? ### Step 0: Check the Active Gateway @@ -83,23 +90,31 @@ Common mistake: - **Laptop-local model + remote gateway**: `host.openshell.internal` points to the remote gateway host, not your laptop. A laptop-local Ollama or vLLM server will not be reachable without a tunnel or shared reachable network path. -### Step 1: Check Whether Managed Inference Is Configured +### Step 1: Check Whether the Relevant Route Is Configured Run: ```bash openshell inference get +openshell inference get --system ``` Interpretation: -- **`Not configured`**: `inference.local` has no backend yet. Fix by configuring it: +- `openshell inference get` shows both the user-facing `inference.local` route and the system route. `--system` isolates the system route. +- **The `inference.local` route is `Not configured`**: managed inference has no backend. Configure it without `--system`: ```bash openshell inference set --provider --model ``` -- **Provider and model shown**: Continue to provider inspection. +- **System inference is `Not configured`**: platform functions have no system backend. Configure it separately: + + ```bash + openshell inference set --system --provider --model + ``` + +- **Provider, model, and timeout shown**: Continue to provider inspection for the relevant route. ### Step 2: Inspect the Provider Record @@ -111,10 +126,13 @@ openshell provider get Check: -- Provider type matches the client API shape +- Provider type matches the client API shape and is supported for managed inference - `openai` for OpenAI-compatible engines such as Ollama, vLLM, SGLang, TRT-LLM, and many NIM deployments - `anthropic` for Anthropic Messages API - `nvidia` for NVIDIA-hosted OpenAI-compatible endpoints + - `deepinfra` for DeepInfra's OpenAI-compatible endpoint + - `google-vertex-ai` for Vertex AI; Claude models use Anthropic Messages and other models use OpenAI Chat Completions + - `aws-bedrock` only through a configured Bedrock-compatible bridge today - Required credential key exists - `*_BASE_URL` override is correct when using a self-hosted endpoint @@ -123,9 +141,11 @@ Fix examples: ```bash openshell provider create --name ollama --type openai --credential OPENAI_API_KEY=empty --config OPENAI_BASE_URL=http://host.openshell.internal:11434/v1 -openshell provider update ollama --type openai --credential OPENAI_API_KEY=empty --config OPENAI_BASE_URL=http://host.openshell.internal:11434/v1 +openshell provider update ollama --credential OPENAI_API_KEY=empty --config OPENAI_BASE_URL=http://host.openshell.internal:11434/v1 ``` +`provider update` preserves the provider type and does not accept `--type`. Prefer bare credential keys, such as `--credential OPENAI_API_KEY`, when reading a real secret from the CLI environment. + ### Step 3: Check Local Host Reachability For host-backed local inference, confirm the upstream server: @@ -142,21 +162,25 @@ Common mistakes: ### Step 4: Check Request Shape -Managed inference only works for `https://inference.local` and supported inference API paths. +User-facing managed inference only works for `https://inference.local` and supported inference API paths. Supported patterns include: - `POST /v1/chat/completions` - `POST /v1/completions` - `POST /v1/responses` +- `POST /v1/embeddings` - `POST /v1/messages` - `GET /v1/models` +- `GET /v1/models/*` +- `POST /model/{modelId}/invoke` for bridge-fronted `aws-bedrock` Common mistakes: - **Wrong scheme**: `http://inference.local` instead of `https://inference.local` - **Unsupported path**: request does not match a known inference API - **Protocol mismatch**: Anthropic client against an `openai` provider, or vice versa +- **Provider-specific mismatch**: Vertex Claude requests must use `/v1/messages`; other Vertex models currently use `/v1/chat/completions`; Bedrock uses its model-in-path invoke shape Fix guidance: @@ -166,6 +190,8 @@ Fix guidance: ### Step 5: Probe from a Sandbox +This probe validates the user-facing `inference.local` route. It does not exercise supervisor-only system inference. + Run a minimal request from inside a sandbox: ```bash @@ -179,21 +205,28 @@ Interpretation: - **`no compatible route`**: provider type and client API shape do not match - **Connection refused / upstream unavailable / verification failures**: base URL, bind address, topology, or credentials are wrong +For system inference failures, inspect the platform function and sandbox supervisor/network logs after confirming `openshell inference get --system`. User code cannot call the `sandbox-system` route directly. + ### Step 6: Reapply or Repair the Managed Route -After fixing the provider, repoint `inference.local`: +After fixing the provider, use `update` for a partial change or `set` to replace the route: ```bash openshell inference set --provider --model +openshell inference update --provider +openshell inference update --model +openshell inference update --timeout 120 ``` +Add `--system` to target the system route. Without it, these commands target `inference.local`. A timeout of `0` uses the 60-second default; increase it for models with long reasoning or idle streaming phases. + If the endpoint is intentionally offline and you only want to save the config: ```bash openshell inference set --provider --model --no-verify ``` -Inference updates are hot-reloaded to all sandboxes on the active gateway within about 5 seconds by default. +Use `--no-verify` only when the endpoint is intentionally offline or the provider protocol cannot be verified, such as the current bridge-fronted Bedrock flow. Inference updates are hot-reloaded to running sandboxes within about 5 seconds by default. ### Step 7: Diagnose Direct External Inference @@ -203,9 +236,31 @@ Check instead: 1. The application is configured to call the external hostname directly 2. A provider with the needed credentials exists -3. The sandbox is launched with that provider attached +3. The sandbox has that provider attached (`openshell sandbox provider list [name]`) 4. `network_policies` allow that host, port, and HTTP rules +If the response reports `credential_endpoint_mismatch`, the provider is attached +but its credential profile does not authorize that request recipient. Run +`openshell provider get ` to identify the provider type, then +inspect its profile endpoints with +`openshell provider profile export -o yaml`. That export uses the current +workspace scope; add `--global` when the provider was created with +`--global-profile`. Compare the profile's endpoint host, port, and path with the +direct request. Correct the provider selection or profile endpoint when that +recipient is intentional. Do not widen the sandbox network policy to work around +the mismatch: policy admission and credential endpoint authorization are +separate checks, and the provider profile should authorize only intended +credential recipients. + +If the response reports `request_authority_mismatch`, compare the HTTP request +authority with the CONNECT tunnel endpoint. The host and effective port must +match. For a tunnel to `api.example.com:8443`, send +`Host: api.example.com:8443`; omitting the non-default port makes the request +authority use the transport default and OpenShell rejects it. An absolute-form +request target must use the same authority. + +Attach or detach a provider on an existing sandbox with `openshell sandbox provider attach ` and `openshell sandbox provider detach `. + Use the `generate-sandbox-policy` skill when the user needs help authoring policy YAML. ## Fix: Local Host Inference Timeouts (Firewall) @@ -305,12 +360,16 @@ Both commands should return the upstream model list. | Symptom | Likely cause | Fix | |---------|--------------|-----| | `openshell inference get` shows `Not configured` | No managed inference route configured | `openshell inference set --provider --model ` | +| System inference is `Not configured` | Platform-only route has no backend | `openshell inference set --system --provider --model ` | | `failed to verify inference endpoint` | Bad base URL, wrong credentials, wrong provider type, or upstream not reachable | Fix provider config, then rerun `openshell inference set`; use `--no-verify` only when the endpoint is intentionally offline | | Base URL uses `127.0.0.1` | Loopback points at the wrong runtime | Use `host.openshell.internal` or another gateway-reachable host | | Local engine works only when gateway is local | Gateway moved to remote host | Run the engine on the gateway host, add a tunnel, or use direct external access | | `connection not allowed by policy` on `inference.local` | Unsupported path or method | Use a supported inference API path | -| `no compatible route` | Provider type does not match request shape | Switch provider type or change the client API | +| `no compatible route` | Provider type does not match request shape | Create or select a provider of the matching type, or change the client API | +| `inference.local` works but a platform function fails | User route is configured but `sandbox-system` is missing or wrong | `openshell inference get --system`; configure or update with `--system`; inspect supervisor logs | | Direct call to external host is denied | Missing policy or provider attachment | Update `network_policies` and launch sandbox with the right provider | +| Direct call returns `credential_endpoint_mismatch` | Attached provider profile does not authorize the request host, port, or path | Inspect the provider profile endpoints; select or update the profile only if it intentionally authorizes that recipient | +| Direct call returns `request_authority_mismatch` | HTTP authority does not match the CONNECT host and effective port | Include the explicit non-default port in `Host` and use the same authority in absolute-form targets | | SDK fails on empty auth token | Client requires a non-empty API key even though OpenShell injects the real one | Use any placeholder token such as `test` | | Upstream timeout from container to host-local backend | Host firewall or network config blocks container-to-host traffic | Allow the Docker bridge subnet to reach the inference port on the host gateway IP (see firewall fix section above) | @@ -328,6 +387,9 @@ openshell gateway info echo "=== Managed Inference ===" openshell inference get +echo "=== System Inference Only ===" +openshell inference get --system + echo "=== Providers ===" openshell provider list @@ -340,7 +402,7 @@ openshell sandbox create -- curl https://inference.local/v1/chat/completions --j When you report back, state: -1. Which inference path is failing (`inference.local` vs direct external) +1. Which inference path is failing (`inference.local`, direct external, or system inference) 2. Whether gateway topology is part of the problem 3. The most likely root cause 4. The exact fix commands the user should run diff --git a/.agents/skills/debug-openshell-cluster/SKILL.md b/.agents/skills/debug-openshell-cluster/SKILL.md index b65bf26d28..17628e980b 100644 --- a/.agents/skills/debug-openshell-cluster/SKILL.md +++ b/.agents/skills/debug-openshell-cluster/SKILL.md @@ -1,11 +1,11 @@ --- name: debug-openshell-cluster -description: Debug why an OpenShell gateway deployment is unhealthy, unreachable, or unable to create sandboxes. Use when the user has a gateway health failure, Docker/Podman runtime issue, Helm install failure, Kubernetes scheduling issue, TLS secret issue, VM driver issue, or sandbox startup problem. Trigger keywords - debug gateway, gateway failing, deployment failing, helm install failing, cluster health, gateway health, gateway not starting, health check failed, sandbox pending, docker driver, podman driver, vm driver. +description: Debug why an OpenShell gateway deployment is unhealthy, unreachable, or unable to create sandboxes. Use for gateway health failures, Docker/Podman runtime issues, Helm failures, Kubernetes scheduling, TLS or auth, gateway interceptors, supervisor middleware startup or runtime failures, external compute-driver sockets, VM drivers, or sandbox startup. Trigger keywords - debug gateway, gateway failing, deployment failing, helm install failing, cluster health, gateway health, gateway not starting, health check failed, sandbox pending, docker driver, podman driver, kubernetes driver, external driver, compute driver socket, gateway interceptor, supervisor middleware, middleware failed, vm driver. --- # Debug OpenShell Gateway Deployment -Diagnose a gateway and its selected compute platform. Do not assume OpenShell provisions Kubernetes or runs a k3s container. OpenShell targets a reachable gateway endpoint backed by Docker, Podman, Kubernetes, or the experimental VM driver. +Diagnose a gateway and its selected compute platform. Do not assume OpenShell provisions Kubernetes or runs a k3s container. OpenShell targets a reachable gateway endpoint backed by Docker, Podman, Kubernetes, the experimental VM driver, or an operator-managed out-of-tree compute driver. Use `openshell` first to identify the active endpoint. Then use the platform tools that match the gateway's compute driver: `docker`, `podman`, `kubectl`/`helm`, or VM driver logs. @@ -15,7 +15,7 @@ The target deployment flow is: 1. Operator starts or deploys the gateway with system packages, systemd, Helm, or a development task. The CLI does not start, stop, or destroy gateway services. 2. Operator configures the compute driver. -3. Operator provides TLS and SSH relay material for the deployment mode. +3. Operator provides the CLI and supervisor authentication material required by the deployment mode: edge or OIDC user auth, optional CLI mTLS, and gateway-minted sandbox JWTs. 4. The CLI registers a reachable gateway endpoint with `openshell gateway add`. 5. The gateway creates sandboxes through the selected compute driver. @@ -25,7 +25,7 @@ For local evaluation only, TLS may be disabled and the gateway can be reached th - The `openshell` CLI must be available for endpoint checks. - Know the active gateway name and endpoint, or be able to inspect local gateway metadata. -- Know the compute platform: Docker, Podman, Kubernetes, or VM. +- Know the compute platform: Docker, Podman, Kubernetes, VM, or an out-of-tree driver. - For Kubernetes: `kubectl` must target the cluster that hosts OpenShell and Helm version 3 or later must be available. - For Docker or Podman: the runtime socket must be reachable from the gateway host. @@ -36,15 +36,24 @@ Run diagnostics in order and stop once the root cause is clear. ### Step 1: Check CLI Reachability ```bash +openshell gateway list --output json openshell gateway info openshell status ``` +For a one-off endpoint check that bypasses stored gateway selection and metadata: + +```bash +openshell --gateway-endpoint status +``` + Common findings: - `No active gateway`: register one with `openshell gateway add `. - Connection refused: gateway process is not running, service exposure is wrong, or a port-forward/proxy is not active. -- TLS/certificate errors: CLI mTLS bundle does not match the gateway CA, or the gateway is running with unexpected TLS settings. +- TLS/certificate errors: the endpoint scheme or trust chain is wrong, a local mTLS bundle does not match the gateway CA, or TLS termination does not match the gateway listener. +- `Unauthenticated` from an edge or OIDC gateway: refresh stored credentials with `openshell gateway login [name]`, then retry. Use `gateway logout` only when intentionally clearing local credentials. +- A direct development endpoint with a private or self-signed certificate can be isolated with `--gateway-endpoint --gateway-insecure`; do not persist or recommend insecure verification for shared gateways. ### Step 2: Identify the Compute Platform @@ -54,10 +63,74 @@ Use gateway metadata, deployment values, or the user's setup notes to identify t |---|---| | Docker | Gateway process logs, Docker daemon health, sandbox containers, image pulls. | | Podman | Podman socket, rootless networking, sandbox containers, image pulls. | -| Kubernetes | Helm release, StatefulSet, service, secrets, sandbox pods, events. | +| Kubernetes | Helm release, gateway workload, service, secrets, sandbox pods, events. | | VM | VM driver logs, rootfs availability, host virtualization support. | +| Extension | External driver process, Unix socket ownership/mode, configured driver name, capability handshake, gateway logs. | + +### Step 3: Check Gateway Startup Dependencies + +Before debugging the compute platform, inspect gateway logs for failures in dependencies initialized before the listener becomes ready. + +For out-of-tree compute drivers, confirm the selected driver name and socket agree across CLI flags or `gateway.toml`, and that the operator-owned driver is running before the gateway starts: + +```bash +rg -n 'compute_drivers|socket_path' /etc/openshell/gateway.toml +stat /run/openshell/.sock +journalctl -u --no-pager --lines=200 +journalctl -u openshell-gateway --no-pager --lines=200 +``` + +Custom names use `[openshell.drivers.].socket_path`. A launch-time `--compute-driver-socket` override may also use `docker`, `podman`, `kubernetes`, or `vm`; the endpoint then takes precedence over built-in construction. The socket must be accessible only to the intended gateway identity. Check gateway logs for connection errors, `GetCapabilities` failures, or an unexpected advertised driver name. The advertised name is diagnostic metadata; negotiated features control optional behavior. The gateway does not create or supervise operator-supplied driver processes or sockets. + +For configured gateway interceptors, inspect `[[openshell.gateway.interceptors]]`, their Unix or network endpoints, and gateway startup logs: + +```bash +rg -n 'interceptors|provider_profile_sources|grpc_endpoint|tls_ca_cert_path|audience|allow_insecure_transport|binding_policy|failure_policy|gateway_jwt' /etc/openshell/gateway.toml +stat /run/openshell/interceptors/.sock +journalctl -u --no-pager --lines=200 +journalctl -u openshell-gateway --no-pager --lines=200 +``` + +The gateway calls each interceptor's `Describe` RPC and validates its manifest at startup. Check for unreachable endpoints, invalid RPC/phase bindings, strict `allowlist` or `exact` mismatches, and `post_commit` bindings that resolve to `fail_closed`. If gateway JWT signing is enabled, authenticated network interceptors require HTTPS and a valid bearer token; check the private CA path, endpoint hostname, expected audience, issuer, `kid`, and interceptor logs for token rejection. `allow_insecure_transport = true` explicitly preserves unauthenticated plaintext behavior. If `provider_profile_sources` names an interceptor, that interceptor must advertise provider-profile capability and return a valid, duplicate-free catalog. A selected interceptor-only source is authoritative; include `builtin` or `user` sources explicitly when composition is intended. + +For operator-run supervisor middleware, inspect `[[openshell.supervisor.middleware]]`, service reachability, and both gateway and supervisor logs: -### Step 3: Check Docker-Backed Gateways +```bash +rg -n 'supervisor|middleware|grpc_endpoint|tls_ca_cert_path|audience|allow_insecure_transport|max_payload_bytes|timeout|gateway_jwt' /etc/openshell/gateway.toml +journalctl -u --no-pager --lines=200 +journalctl -u openshell-gateway --no-pager --lines=200 +openshell logs --tail --source sandbox +``` + +The middleware service must start before the gateway and be reachable from both the gateway and sandbox supervisors. Gateway startup fails if `Describe` is unavailable, a manifest exposes duplicate operation/phase bindings, the registration claims the reserved `openshell/` namespace, or payload and timeout limits are invalid. Supported V1 bindings are `HTTP_REQUEST/PRE_CREDENTIALS` and `WEBSOCKET_MESSAGE/PRE_CREDENTIALS`. When gateway JWT signing is disabled, supervisors preserve the legacy unauthenticated connector and do not request extension credentials. When signing is enabled, credential acquisition and verification failures are fail closed: check HTTPS trust and hostname validation, audience and issuer agreement, the token `kid`, gateway `RefreshSandboxToken` errors, and middleware logs. Changing a registration requires a gateway restart. A policy update can also fail before persistence if the selected implementation rejects its `network_middlewares` config. + +At request time, distinguish attachment, binding selection, coverage, denial, and failure. A host-matched HTTP-only attachment can inspect the upgrade GET but does not join the WebSocket chain; the connection proceeds under either `on_error` mode and emits `binding_not_selected` coverage. A selected WebSocket stage receives text messages only. Binary messages pass under both modes, emit `unsupported_message_type` coverage, and consume a session sequence without an RPC. An explicit `middleware_denied` result is always enforced. WebSocket preflight returns `INSPECT`, voluntary `SKIP`, or authoritative `DENY`; `DENY` rejects the upgrade before upstream contact under both `on_error` modes. A selected-stage failure follows the policy-local `on_error`: `fail_closed` blocks the HTTP request or closes the WebSocket, while `fail_open` bypasses only that stage and emits a detection finding. A fail-open per-message capacity failure bypasses that message without disabling the stage. A timeout, transport failure, stream closure, missing or invalid response, duplicate or regressed sequence, or other failure that makes an established WebSocket stream unreliable disables that stage for later messages on the connection and emits `openshell.middleware.websocket_stage_disabled`. Confirm preflight, session-start, and session-end in service logs. OpenShell best-effort sends at most one session-end to each still-writable opened stage, including a preflight that terminates before session start; distinguish `MIDDLEWARE_DENIAL` from `MIDDLEWARE_FAILURE`. WebSocket message sequences are allocated session-wide; each stage receives a strictly increasing subset, so gaps are valid when binary messages or other units are not delivered to that stage. Zero, duplicate, or regressed sequences are protocol errors. If a running supervisor cannot install a new registry, it preserves its last-known-good generation and emits a configuration failure event. + +For network policy validation failures, first distinguish a gateway mutation +rejection from a supervisor runtime rejection. Direct policy updates, +incremental merges and approvals, provider attachments, and provider-profile +fanout are validated against the complete effective policy before persistence +when the gateway knows the affected sandbox scope. A `FAILED_PRECONDITION` +ambiguity response means no invalid revision or partial fanout was stored. +Supervisor validation remains defense in depth for startup, races, and policy +sources outside those mutation paths. + +Runtime rejection behavior is configured only in `gateway.toml`: + +```toml +[openshell.gateway] +policy_validation_failure_mode = "fail_closed" +``` + +The default `fail_closed` mode deactivates the previous generation, closes +pinned relays, and quarantines new egress until a valid generation loads. +`retain_last_valid` explicitly keeps the previous valid policy active; without +one it still fails closed. Restart the gateway after changing this field. +Inspect sandbox OCSF configuration and finding events for the validation +rationale, configured and effective modes, active generation, and the explicit +`previous_policy_active` state. + +### Step 4: Check Docker-Backed Gateways ```bash docker info @@ -99,8 +172,19 @@ Common findings: - Docker daemon unavailable: start Docker Desktop or Docker Engine. - Gateway process stopped: inspect exit status and logs. - Sandbox image missing or pull denied: verify image reference and registry credentials. +- Sandbox fails before readiness with an identity-resolution error: inspect the image's OCI `USER` and matching `/etc/passwd` and `/etc/group` entries, or explicitly set both process identity fields in policy. Numeric workload identities `1` through `4294967294` are accepted; root, the invalid identity sentinel, and missing identities are rejected. +- Sandbox fails before readiness with an OCI workspace validation error: inspect the image's `WorkingDir` using the immutable image ID reported by the gateway. Empty, `/`, and explicit `/sandbox` use the managed `/sandbox` compatibility workspace. Any other workdir must be an absolute normalized directory with no symlink components; the final policy UID, primary GID, and supplementary groups must pass the kernel's effective traverse/write checks, including POSIX ACL and LSM decisions. OpenShell does not create, chown, or chmod a non-default image workdir. +- Docker also rejects an image `VOLUME` that covers the workdir or one of its parents because the runtime would mask the immutable path before validation. Move the `VOLUME` below the workspace or remove the declaration. +- A workdir rejected as a special filesystem or OpenShell control-path collision cannot be made valid with permissions. Move the image workdir away from kernel-backed mounts and the concrete supervisor, TLS, token, runtime, and socket paths named in the error. - Docker driver cannot initialize because it cannot find `openshell-sandbox`: verify `OPENSHELL_DOCKER_SUPERVISOR_BIN`, the sibling binary next to `openshell-gateway`, or the configured supervisor image contains `/openshell-sandbox`. - Sandbox never registers: check gateway logs and supervisor callback endpoint. +- On macOS, repeated `Policy fetch failed after 5 attempts` messages with a + Homebrew gateway bound to `[::1]:17670` indicate that the Docker + `host-gateway` IPv4 route has no matching callback listener. Current releases + leave `bind_address` unset in the Homebrew config, use the built-in + `127.0.0.1:17670` primary listener, and reuse it for authenticated sandbox + callbacks. On an older release, set `bind_address = "127.0.0.1:17670"` or + upgrade. - Supervisor image exits before printing `openshell-sandbox --version`: the image should be the scratch supervisor image from `deploy/docker/Dockerfile.supervisor` and must contain a static executable at `/openshell-sandbox`. - `mise run e2e:docker:gpu` fails with `docker info --format json did not report any discovered NVIDIA CDI GPU devices`: Docker may report `CDISpecDirs` while still having no generated NVIDIA CDI specs. Verify `.DiscoveredDevices` contains entries such as `nvidia.com/gpu=all`, verify `/etc/cdi` or `/var/run/cdi` contains a generated NVIDIA spec, and check that `nvidia-cdi-refresh.service` and `nvidia-cdi-refresh.path` from NVIDIA Container Toolkit are enabled and healthy. The service is a one-shot unit, so `inactive (dead)` can be normal after a successful run; use `systemctl status` and `journalctl` to distinguish success from a skipped or failed refresh. NVIDIA recommends enabling the path and service units, and restarting `nvidia-cdi-refresh.service` to regenerate missing or stale CDI specs. If specs are generated but Docker still reports no discovered devices, restart Docker or reload the daemon and re-check `docker info`. @@ -110,7 +194,16 @@ For source checkout development, restart the local gateway with: mise run gateway:docker ``` -### Step 4: Check Podman-Backed Gateways +During a graceful gateway restart, Docker, Podman, and VM sandboxes with +running intent should stop before the gateway exits and restart after it +returns. Check for `Stopped sandbox during gateway shutdown` and `Started +sandbox during gateway startup` in gateway logs. A sandbox explicitly stopped +through the CLI remains stopped. Kubernetes sandboxes are cluster-owned and do +not follow this local gateway lifecycle. Internal and external drivers follow +the same rule: `GetCapabilities.gateway_manages_lifecycle` must be true for the +gateway to run shutdown and startup sweeps. + +### Step 5: Check Podman-Backed Gateways ```bash podman info @@ -124,27 +217,77 @@ Common findings: - Podman socket unavailable: start or expose the user socket. - Rootless networking unavailable: inspect Podman network configuration. - Sandbox image missing or pull denied: verify image reference and registry credentials. +- Sandbox fails before readiness with an identity-resolution error: inspect the image's OCI `USER` and matching `/etc/passwd` and `/etc/group` entries, or explicitly set both process identity fields in policy. Numeric workload identities `1` through `4294967294` are accepted; root, the invalid identity sentinel, and missing identities are rejected. - Supervisor cannot call back: check callback endpoint and gateway logs. - -### Step 5: Check Kubernetes Helm Gateways +- Gateway exits before becoming healthy with a callback-listener discovery + error: inspect `podman info --debug`, the configured Podman network, and the + host's IPv4 default route. Rootless pasta uses the private source address + selected by that route; rootful Podman uses the bridge gateway address. +- Current gateways reuse the primary listener when it covers Podman's callback + address. If the primary does not cover that address, inspect the gateway + startup logs for the additional callback-only listener and its provenance. +- Rootless slirp4netns, another named helper, or missing helper metadata + requires an explicitly remote `grpc_endpoint`. An explicit `host_gateway_ip` + cannot bypass slirp4netns host-loopback isolation. Do not work around + discovery failures by broadening the primary gateway listener to `0.0.0.0`. + +When `userns` is configured (e.g. `userns = "auto"` or `userns = "keep-id"`): + +- Supervisor delivery uses bind-mount fallback instead of image volumes because + overlay mounts do not support `idmapped` mounts. The supervisor binary is + extracted from the supervisor image and cached at + `$XDG_DATA_HOME/openshell/podman-supervisor/` (typically + `~/.local/share/openshell/podman-supervisor/`). +- Stale cache: if the supervisor image is updated but the cached binary is not + refreshed, sandbox creation may fail with an ELF validation error or version + mismatch. Remove the cache directory and retry. +- `auto` mode requires subuid/subgid ranges for the current user in + `/etc/subuid` and `/etc/subgid`. If missing, Podman returns a user-namespace + mapping error at container creation. +- `private` mode requires explicit `uidmap` and `gidmap` arrays in the TOML + config. Without both, the gateway rejects the config at startup. + Rootless Podman uses intermediate IDs (e.g. `uidmap = ["0:0:1", "1:1:65535"]`); + rootful Podman uses absolute host IDs (e.g. `uidmap = ["0:1000:1", "1:100000:65536"]`). +- `nomap` (without hyphen) is accepted as input but canonicalized to `no-map` + for Podman's API. + +### Step 6: Check Kubernetes Helm Gateways ```bash helm -n openshell status openshell helm -n openshell get values openshell -kubectl -n openshell get statefulset,pod,svc,pvc -kubectl -n openshell logs statefulset/openshell --tail=200 +kubectl -n openshell get deployment,statefulset,pod,svc,pvc +kubectl -n openshell logs deployment/openshell -c openshell-gateway --tail=200 +kubectl -n openshell logs statefulset/openshell -c openshell-gateway --tail=200 +kubectl -n openshell rollout status deployment/openshell kubectl -n openshell rollout status statefulset/openshell ``` -Look for failed installs, unexpected values, missing namespace, wrong image tag, TLS settings that do not match the registered endpoint, and scheduling failures. +Use the log and rollout commands for the workload kind that exists in the +release. Look for failed installs, unexpected values, missing namespace, wrong +image tag, TLS settings that do not match the registered endpoint, and +scheduling failures. + +`server.telemetryEnabled` renders `OPENSHELL_TELEMETRY_ENABLED` on the gateway +pod, and the gateway propagates the effective value to sandbox supervisors. -For HA or PostgreSQL-backed installs, also check the service-binding Secret and -bundled PostgreSQL workload: +When no external credential driver is enabled, the Helm chart uses the +gateway's default encrypted database credential storage. The chart creates a +retained Kubernetes Secret for the shared KEK, injects it into gateway pods, and +stores encrypted credential envelopes in the OpenShell database. For +`workload.kind=deployment` or multi-replica gateways, confirm +`server.externalDbSecret` points at a shared database. A render/install error +mentioning `server.credentialDrivers` means the values selected multiple +external credential backends. + +For HA or PostgreSQL-backed installs, also check the external database Secret +referenced by `server.externalDbSecret` and the PostgreSQL workload if the test +or operator deployed one in-cluster: ```bash -kubectl -n openshell get secret -l app.kubernetes.io/instance=openshell -kubectl -n openshell get statefulset,pod,pvc -l app.kubernetes.io/instance=openshell -kubectl -n openshell logs statefulset/openshell-postgres --tail=200 +kubectl -n openshell get secret openshell-ha-pg -o yaml +kubectl -n openshell get deployment,service,pod -l app.kubernetes.io/name=openshell-e2e-postgres +kubectl -n openshell logs deployment/openshell-e2e-postgres --tail=200 ``` Check required Helm deployment secrets: @@ -168,15 +311,80 @@ Secrets but does not create the sandbox JWT signing Secret. If the gateway exits with `failed to read sandbox JWT signing key from /etc/openshell-jwt/signing.pem`, verify that `openshell-jwt-keys` contains -`signing.pem`, `public.pem`, and `kid`, and that the StatefulSet mounts the +`signing.pem`, `public.pem`, and `kid`, and that the gateway workload mounts the `sandbox-jwt` secret at `/etc/openshell-jwt`. The sandbox JWT mount is required even when local Helm values disable TLS. +If `certManager.serverIssuerRef` points the server certificate at an external +Issuer or ClusterIssuer (for example an ACME issuer, for a publicly-trusted +cert on an OpenShift `Route` with TLS passthrough — see +`openshiftRoute.enabled`), the chart creates **two** server certificates: an +internal one (chart CA, internal SANs) and an external one (from the configured +issuer, external SANs only). The gateway uses SNI to present the right cert. + +Check the external `Certificate`/`CertificateRequest`/`Challenge` resources +directly when the external secret never becomes Ready: + +```bash +kubectl -n openshell get certificate,certificaterequest,challenge +kubectl -n openshell describe certificate openshell-server-external +oc -n openshell get route +``` + +ACME issuers reject certificate requests that include internal-only names +(`*.svc.cluster.local`, `localhost`, loopback IPs) and require the +`commonName` to also be a SAN — the external `Certificate` only requests the +hostnames in `certManager.serverDnsNames`, for exactly this reason. + +If sandbox supervisors fail their TLS handshake to the gateway with +`UnknownCA` after configuring `serverIssuerRef`, the most likely cause is +`server.grpcEndpoint` set to the external hostname. This forces supervisors +to connect via the external hostname, receiving the ACME cert (via SNI) which +they cannot verify against the chart CA. Remove `server.grpcEndpoint` or set +it to the internal service name so supervisors receive the internal cert: + +```bash +helm -n openshell get values openshell | grep -E 'grpcEndpoint|clientCaFromServerTlsSecret|clientCaSecretName|serverIssuerRef|caSecretName' +# server.grpcEndpoint should be unset or point to internal service name +``` + +Less commonly, `UnknownCA` can occur if the gateway's client-verification CA +is misconfigured. The default `clientCaFromServerTlsSecret=true` is correct +for all configurations — the internal server certificate is always signed by +the chart CA (the same CA that signs the client cert), so its `ca.crt` is +the right trust anchor. Only override this if you intentionally mount a +separate client CA via `server.tls.clientCaSecretName`. Verify the mounted +client CA matches the CA that signed the client certificate: + +```bash +kubectl -n openshell get statefulset openshell -o jsonpath='{.spec.template.spec.volumes[?(@.name=="tls-client-ca")]}' | jq . +# Should show items filter for ca.crt from openshell-server-tls +``` + +If `server.providerTokenGrants.spiffe.enabled=true`, the gateway should still +render `[openshell.gateway.gateway_jwt]` and mount the `sandbox-jwt` Secret. +SPIRE is used only by sandbox pods for dynamic provider token grants. Verify +that SPIRE is installed, the CSI driver is available, and the Kubernetes driver +config includes `provider_spiffe_workload_api_socket_path`: + +```bash +helm -n openshell get values openshell | grep -E 'providerTokenGrants|workloadApiSocketPath' +kubectl get pods -A | grep -E 'spire|spiffe' +kubectl -n openshell get configmap openshell-config -o yaml | grep provider_spiffe_workload_api_socket_path +``` + +Sandbox pods using provider token grants should have an +`openshell.io/sandbox-id` annotation, an `openshell.ai/managed-by=openshell` +label, supervisor env vars `OPENSHELL_K8S_SA_TOKEN_FILE` and +`OPENSHELL_PROVIDER_SPIFFE_WORKLOAD_API_SOCKET`, plus both the projected +`openshell-sa-token` volume and the `spiffe-workload-api` CSI volume. + Check the image references currently used by the gateway deployment: ```bash +kubectl -n openshell get deployment openshell -o jsonpath="{.spec.template.spec.containers[*].image}{\"\n\"}{.spec.template.spec.containers[*].env[?(@.name==\"OPENSHELL_SUPERVISOR_IMAGE\")].value}{\"\n\"}" kubectl -n openshell get statefulset openshell -o jsonpath="{.spec.template.spec.containers[*].image}{\"\n\"}{.spec.template.spec.containers[*].env[?(@.name==\"OPENSHELL_SUPERVISOR_IMAGE\")].value}{\"\n\"}" -helm -n openshell get values openshell | grep -E 'repository|tag|supervisorImage' +helm -n openshell get values openshell | grep -E 'repository|tag|supervisorImage|workload' ``` The gateway image built from `deploy/docker/Dockerfile.gateway` and the scratch supervisor image built from `deploy/docker/Dockerfile.supervisor` should use the same build tag in branch and E2E deploys. A stale supervisor image can make sandbox behavior lag behind gateway policy or proto changes. @@ -219,7 +427,8 @@ If the gateway is healthy but sandbox creation fails: ```bash kubectl -n openshell get pods kubectl -n openshell get events --sort-by=.lastTimestamp | tail -n 50 -kubectl -n openshell logs statefulset/openshell --tail=200 +kubectl -n openshell logs deployment/openshell -c openshell-gateway --tail=200 +kubectl -n openshell logs statefulset/openshell -c openshell-gateway --tail=200 ``` Check the configured sandbox namespace: @@ -242,7 +451,110 @@ kubectl -n openshell get configmap openshell-config -o jsonpath='{.data.gateway\ kubectl -n get sandbox -o jsonpath='{.spec.template.spec.serviceAccountName}{"\n"}' ``` -### Step 6: Check VM-Backed Gateways +If `topology = "sidecar"` is rendered under `[openshell.drivers.kubernetes]`, +sandbox pods should have an `openshell-network-init` init container running +`--mode=network-init`, an `agent` container running +`openshell-sandbox --mode=process`, and an `openshell-supervisor-network` +container running `--mode=network`. The init container owns nftables setup and +should be the only sidecar topology container with `NET_ADMIN`. It also needs +`CHOWN`/`FOWNER` to hand shared emptyDir state to the effective sidecar UID. The +default binary-aware network sidecar runs as UID 0 with primary GID +`sandbox_gid` and adds `SYS_PTRACE` plus `DAC_READ_SEARCH`. When +`process_binary_aware_network_policy = false`, it runs as the configured +non-root `proxy_uid` without those inspection capabilities. That dedicated +proxy UID must remain at least `1000` and must not match the workload UID +because the pod egress fence exempts its traffic. The pod `fsGroup` is set to +`sandbox_gid` in both modes. + +In sidecar topology only the network sidecar should mount the gateway bootstrap +credentials (`openshell-sa-token` and `openshell-client-tls`). The process +container should not receive `OPENSHELL_ENDPOINT`, gateway TLS env vars, the +sandbox token file, or those credential mounts. Instead, the network sidecar +serves policy and provider environment state over the Unix control socket from +`OPENSHELL_SIDECAR_CONTROL_SOCKET` (`/run/openshell-sidecar/control.sock` by +default). The process supervisor must be the first and only client. After +validating its peer UID, GID, and PID, the sidecar unlinks the listener. If the +connection later closes, the network sidecar exits non-zero so Kubernetes can +restart it with a fresh listener. If the process supervisor fails before +launching the workload, +inspect both containers for control-socket bind, connect, bootstrap, or update +errors. If new SSH/exec sessions do not pick up refreshed provider environment, +inspect the network sidecar settings-poll logs and the process container logs +for provider environment update handling; the process container should consume +newer provider-env revisions without receiving gateway credentials. + +The process container reports the workload entrypoint PID over the same control +socket, and the network sidecar uses that PID for binary-scoped policy +decisions through `/proc`. If rules with `policy.binaries` are unexpectedly +denied, inspect the sidecar control logs and confirm the pod has +`shareProcessNamespace: true`. +The shared state directory should preserve `sandbox_gid` inheritance +(`02775`). Sidecar SSH uses the Linux abstract socket +`@openshell-sidecar-ssh`; the network sidecar verifies its peer PID before +bridging gateway relay requests. No `ssh.sock` file should appear in the shared +state directory. +Inspect all three when sandbox registration or egress enforcement fails: + +```bash +kubectl -n openshell get configmap openshell-config -o jsonpath='{.data.gateway\.toml}' | grep -E '^\[openshell\.drivers\.kubernetes\]|^topology\s*=' +kubectl -n get pod -o jsonpath='{range .spec.initContainers[*]}{.name}{" "}{.command}{"\n"}{end}' +kubectl -n get pod -o jsonpath='{range .spec.containers[*]}{.name}{" "}{.command}{"\n"}{end}' +kubectl -n logs -c openshell-network-init --tail=200 +kubectl -n logs -c openshell-supervisor-network --tail=200 +kubectl -n logs -c agent --tail=200 +``` + +#### Corporate upstream proxy + +When the deployment routes sandbox egress through a corporate HTTP forward +proxy, the operator-owned settings render under `[openshell.drivers.kubernetes]` +from the Helm `upstreamProxy` values. Absent proxy configuration preserves +direct-dial egress; any present-but-invalid value fails closed at gateway +startup (`validate_upstream_proxy_config`) rather than silently reverting to a +direct connection. Confirm the rendered configuration first: + +```bash +kubectl -n openshell get configmap openshell-config -o jsonpath='{.data.gateway\.toml}' | grep -E 'https_proxy|no_proxy|proxy_auth_secret_(name|key)|proxy_auth_allow_insecure|proxy_connect_by_hostname' +helm -n openshell get values openshell | grep -A8 upstreamProxy +``` + +Only `http://host:port` forward proxies are supported; `https://` proxy URLs and +plain-HTTP egress are out of scope and rejected. Proxy credentials require +`topology = "sidecar"` — combined topology shares the credential mount with the +workload, so the gateway rejects credentials there. The credential Secret named +by `proxy_auth_secret_name` must exist in the sandbox namespace with the key +named by `proxy_auth_secret_key`, and Kubernetes will not create keys longer +than 253 bytes or named `.`/`..`. + +The proxy arguments and credential mount are injected only into the container +that runs network supervision (the `agent` container in combined topology, the +`openshell-supervisor-network` sidecar in sidecar topology). The one-shot +`openshell-network-init` container and the process `agent` container in sidecar +topology must never receive them. The credential is projected read-only as the +`openshell-upstream-proxy-auth` volume at `/run/openshell/upstream-proxy-auth` +and passed as `--upstream-proxy-auth-file`; it must never appear in env, +annotations, or command arguments. + +```bash +kubectl -n get secret -o jsonpath='{.data}' >/dev/null && echo "secret present" +kubectl -n get pod -o jsonpath='{range .spec.containers[*]}{.name}{" "}{.command}{"\n"}{end}' | grep -- '--upstream-' +kubectl -n get pod -o jsonpath='{range .spec.containers[*]}{.name}{": "}{range .volumeMounts[*]}{.name}{" "}{end}{"\n"}{end}' | grep upstream-proxy-auth +kubectl -n get events --sort-by=.lastTimestamp | grep -Ei 'secret|MountVolume' | tail -n 20 +``` + +A missing Secret or wrong key leaves the pod stuck with a +`MountVolume.SetUp failed` / `secret ... not found` event. If the pod starts but +egress still fails, the corporate proxy itself is the next suspect: policy- +approved TLS CONNECT requests that time out after policy evaluation usually mean +the proxy URL is unreachable from the sandbox namespace, or a cluster-internal +destination that should be direct is missing from `no_proxy`. Inspect the +network supervisor logs for CONNECT and upstream-proxy decisions: + +```bash +kubectl -n logs -c openshell-supervisor-network --tail=200 | grep -Ei 'upstream|connect|proxy' +``` + +### Step 7: Check VM-Backed Gateways Use the VM driver logs and host diagnostics available in the user's environment. Verify: @@ -264,13 +576,35 @@ openshell logs |---|---|---| | `openshell status` fails | Gateway endpoint unreachable or auth mismatch | `openshell gateway info`, gateway logs | | Gateway starts but sandbox create fails | Compute driver cannot reach runtime | Docker/Podman/Kubernetes/VM driver logs | +| Gateway exits while resolving compute-driver listener requirements | Callback alias topology is unsupported, the Podman network cannot be inspected, or the selected address is not private/authorized | Gateway startup error, `podman info --debug`, Podman network inspection, host IPv4 default route | +| Admin, health, reflection, or HTTP request is denied on an additional Docker/Podman callback-only listener | Additional callback listeners intentionally expose only sandbox-callable gRPC methods | Retry through the gateway's primary endpoint; inspect the listener-purpose startup log if the address was unexpected | | Docker or Podman sandbox never registers | Wrong callback endpoint or supervisor startup failure | Gateway logs and sandbox container logs | | Docker GPU e2e fails before GPU sandbox comparison | NVIDIA CDI specs are missing or Docker has not discovered them | `docker info --format '{{json .DiscoveredDevices}}'`, `/etc/cdi`, `/var/run/cdi`, `nvidia-cdi-refresh.service` | | Kubernetes gateway pod pending | PVC unbound, taint, selector, or insufficient resources | `kubectl -n openshell describe pod ` | -| Kubernetes gateway pod crash loops | Missing secret, bad DB URL, bad TLS config | `kubectl -n openshell logs statefulset/openshell` | +| Kubernetes sandbox pod stuck pending, workspace PVC unbound | Cluster has no default `StorageClass` and OpenShell does not set `storageClassName` on the workspace PVC (clusters with a default `StorageClass` bind fine without it) | `kubectl -n openshell describe pvc`; set `server.workspaceStorageClass` (gateway config `workspace_storage_class`) to a valid `StorageClass` | +| Kubernetes gateway pod crash loops | Missing secret, bad DB URL, bad TLS config | `kubectl -n openshell logs deployment/openshell -c openshell-gateway` or `kubectl -n openshell logs statefulset/openshell -c openshell-gateway` | | CLI TLS error | Local mTLS bundle does not match server cert/CA | Check `~/.config/openshell/gateways//mtls/` | +| Edge or OIDC gateway returns `Unauthenticated` | Stored login expired, audience/scopes mismatch, or gateway auth configuration changed | `openshell gateway info`, `openshell gateway login `, gateway auth logs | +| Gateway fails before serving health after enabling an interceptor | Interceptor endpoint unavailable or manifest/binding validation failed | Gateway and interceptor logs; interceptor socket; `binding_policy`, phases, and failure policy | +| Authenticated interceptor or middleware rejects gateway calls | Private CA or hostname mismatch, expected audience or issuer mismatch, stale/unknown `kid`, or malformed extension token | `tls_ca_cert_path`, registration `audience`, service verifier config and logs; fetch well-known metadata only through the already-trusted gateway TLS endpoint | +| Provider profiles disappear after enabling an interceptor catalog | `provider_profile_sources` selected only an authoritative interceptor or returned invalid/duplicate IDs | Inspect source list and interceptor `Describe`/catalog logs; include `builtin` and `user` when intended | +| Gateway fails after registering supervisor middleware | Service unavailable, invalid manifest, duplicate binding, reserved name, or invalid payload/timeout limit | Middleware service and gateway logs; `[[openshell.supervisor.middleware]]`; `Describe` response | +| Policy update rejects `network_middlewares` | Unknown middleware name, implementation-owned config invalid, duplicate order, broad/invalid host selector, or fail-closed coverage of `tls: skip` | Policy error, gateway logs, middleware `ValidateConfig`, selector and order fields | +| Policy mutation returns `FAILED_PRECONDITION` for endpoint ambiguity | Equally specific effective endpoint selectors disagree on connection or request-processing metadata | CLI error, base and provider-composed policy, affected profile attachments; confirm no new revision was stored | +| Supervisor enters policy quarantine | A runtime candidate failed validation while `policy_validation_failure_mode = "fail_closed"` | Sandbox OCSF config/finding events, validation rationale, active generation, `previous_policy_active` | +| HTTP request returns `middleware_failed` or `middleware_denied`, or WebSocket closes with `1008` | Selected stage failed or explicitly denied admitted traffic | Sandbox OCSF logs; policy-local middleware config; service availability; binding operation; `on_error` | +| WebSocket upgrades but a host-matched middleware receives no preflight or message RPC | The implementation did not advertise `WEBSOCKET_MESSAGE/PRE_CREDENTIALS` | `WEBSOCKET_MIDDLEWARE_COVERAGE state=binding_not_selected`; service `Describe`; the upgrade GET may still have used its HTTP binding | +| Binary WebSocket message passes without a middleware RPC | Binary is unsupported by the V1 text-message binding under both `on_error` modes | `WEBSOCKET_MIDDLEWARE_COVERAGE state=unsupported_message_type`; the next text RPC may have a valid sequence gap | +| WebSocket messages stop reaching middleware after one failure | A fail-open stage stream was disabled for the rest of the connection | `openshell.middleware.websocket_stage_disabled`; middleware timeout/stream/protocol logs. A per-message capacity bypass alone leaves the stage active. Reconnect to create a fresh stream after a genuine stream failure | +| Supervisor repeatedly fails to install middleware after enabling gateway JWT signing | Extension credential minting, distribution, or authenticated service connection failed; last-known-good registry remains active | Gateway `RefreshSandboxToken` logs, sandbox configuration events, service token-verification logs, registration TLS/audience settings | +| Custom compute driver is unavailable | Driver process/socket missing, inaccessible, or selected name does not match its endpoint/config key | Socket ownership/mode, driver service logs, gateway `GetCapabilities` logs | +| Sandbox remains `Stopping` or `Starting` | Driver stop/start failed, retained resource is missing, or a fresh supervisor has not connected | Gateway and driver logs; `docker inspect`, `podman inspect`, Agent Sandbox status/PVC, or VM state marker and launcher process | | Image pull failure | Gateway or sandbox image cannot be pulled | Runtime events and image pull credentials | | `K8s namespace not ready` with `envoy-gateway-openshell.yaml: the server could not find the requested resource` | Optional Gateway API manifest was applied without Envoy Gateway CRDs, or k3s Helm controller startup exceeded the namespace wait | Apply `deploy/kube/manifests/envoy-gateway-openshell.yaml` manually only after Envoy Gateway is installed and `grpcRoute` is enabled | +| HTTPS ingress (`grpcRoute.gateway.listener.protocol=HTTPS`) connection resets or TLS handshake hangs | Envoy terminates TLS but the gateway pod still expects TLS, so the plaintext backend hop fails | Set `server.disableTls=true` so Envoy forwards plaintext to the pod; verify the listener `certificateRefs` Secret exists in the release namespace and `openshell status` over `https://` | +| HTTPS ingress returns `Unauthenticated` after connecting | TLS terminates at Envoy, so the gateway never sees a client cert; no OIDC issuer is configured for identity | Configure `server.oidc.issuer` and register with `openshell gateway add https:// --oidc-issuer `, or set `server.auth.allowUnauthenticatedUsers=true` for a trusted-proxy/dev cluster | +| External server `Certificate` never becomes Ready with `certManager.serverIssuerRef` set | ACME issuer rejected internal-only SANs, a loopback IP, or a `commonName` absent from the SANs | `kubectl -n openshell describe certificate openshell-server-external`; confirm `certManager.serverDnsNames` lists only real, externally-resolvable hostnames | +| Sandbox supervisors fail TLS handshake with `UnknownCA` after configuring `certManager.serverIssuerRef` | `server.grpcEndpoint` is set to the external hostname, forcing supervisors to receive the ACME cert (via SNI) which they can't verify against chart CA | Remove `server.grpcEndpoint` or set it to the internal service name; supervisors should connect via internal service name to receive the internal cert | ## Reporting @@ -280,7 +614,7 @@ When handing results back to the user, include: - Compute platform and driver. - Gateway process or workload status. - Recent gateway log summary. -- Missing or malformed TLS or SSH relay material. +- Missing or malformed TLS, OIDC/mTLS, or sandbox JWT material. - Service exposure status. - Sandbox workload status. - The exact command that failed and the shortest fix. diff --git a/.agents/skills/fix-security-issue/SKILL.md b/.agents/skills/fix-security-issue/SKILL.md index 75703c4bfb..4e6610c8a1 100644 --- a/.agents/skills/fix-security-issue/SKILL.md +++ b/.agents/skills/fix-security-issue/SKILL.md @@ -1,6 +1,6 @@ --- name: fix-security-issue -description: Implement a fix for a reviewed security issue. Takes an issue number or scans for issues labeled "topic:security" and "state:agent-ready". Reads the security review from the issue comments and implements the remediation plan. Trigger keywords - fix security issue, remediate security, implement security fix, patch vulnerability. +description: Implement a fix for a reviewed security issue. Takes a directly requested issue number or scans for issues labeled `topic:security` and `agent:implementation-requested`. Reads the security review from the issue comments and implements the remediation plan. Trigger keywords - fix security issue, remediate security, implement security fix, patch vulnerability. --- # Fix Security Issue @@ -11,7 +11,7 @@ Implement a code fix for a security issue that has already been reviewed by the - The `gh` CLI must be authenticated (`gh auth status`) - You must be in a git repository with a GitHub remote -- The issue **must** have both the `topic:security` and `state:agent-ready` labels. If either is missing, do not proceed. +- The issue must have `topic:security`. In unattended scan mode it must also have `agent:implementation-requested`; a direct user request to fix a specific issue does not require that label. - The issue must have a prior security review comment (posted by `review-security-issue`) with a **Legitimate concern** determination and a remediation plan ## Agent Comment Marker @@ -30,14 +30,14 @@ The user may provide an issue number directly, or ask the agent to find issues t ### If an issue number is provided -Strip any leading `#` and proceed to Step 2 with that issue ID. +Strip any leading `#` and proceed to Step 2 with that issue ID. The user's explicit fix request authorizes implementation; do not refuse solely because `agent:implementation-requested` is absent. ### If no issue number is provided -Scan for open issues labeled `topic:security` and `state:agent-ready`: +Scan for open issues labeled `topic:security` and `agent:implementation-requested`: ```bash -gh issue list --label "topic:security" --label "state:agent-ready" --state open --json number,title,labels,updatedAt +gh issue list --label "topic:security" --label "agent:implementation-requested" --state open --json number,title,labels,updatedAt ``` - **If no issues are found**, report to the user that there are no security issues ready for fixing and stop. @@ -52,20 +52,16 @@ Fetch the issue details: gh issue view --json number,title,body,state,labels,author ``` -### Require both `topic:security` and `state:agent-ready` labels +### Validate the Security Label and Invocation Mode -**This is a hard gate.** Check the issue's `labels` array from the response above. Both of the following labels **must** be present: +Check the issue's `labels` array from the response above: -- `topic:security` -- `state:agent-ready` +- `topic:security` is required because this specialized skill handles security issues. +- `agent:implementation-requested` is required only when an unattended agent discovered the issue by scanning the queue. -If **either label is missing**, do **not** proceed. Report to the user which label(s) are missing and stop. For example: +If `topic:security` is missing, report that this skill only handles security issues and stop. If queue mode selected an issue without `agent:implementation-requested`, report that it is not ready for unattended pickup and stop. -- Missing `state:agent-ready`: "Issue #42 has the `topic:security` label but is not marked `state:agent-ready`. It may still need review or human triage before a fix can be implemented." -- Missing `topic:security`: "Issue #42 is marked `state:agent-ready` but does not have the `topic:security` label. This skill only handles security issues." -- Missing both: "Issue #42 is missing both the `topic:security` and `state:agent-ready` labels. Cannot proceed." - -**Do not offer to add the labels or bypass this check.** The labels are a deliberate human-controlled gate. +Never apply `agent:implementation-requested` yourself. Its absence does not block a direct user request to fix a specific issue. ### Validate the security review @@ -100,6 +96,12 @@ git checkout -b fix/security-- Follow the project's branch naming conventions. The branch name should reference the issue ID. +In queue mode, replace the human request and ready-plan labels with the agent execution state. For an unlabeled direct invocation, do not add an agent-workflow label: + +```bash +gh issue edit --remove-label "agent:implementation-requested" --remove-label "agent:plan-ready" --add-label "agent:in-progress" +``` + ## Step 5: Implement the Fix Implement the changes described in the remediation plan. Follow these principles: @@ -232,6 +234,12 @@ EOF Created PR [#](https://github.com/OWNER/REPO/pull/) ``` +In queue mode, replace `agent:in-progress` with `agent:pr-opened` after the PR is created. For an unlabeled direct invocation, do not add an agent-workflow label: + +```bash +gh issue edit --remove-label "agent:in-progress" --add-label "agent:pr-opened" +``` + ## Step 9: Report to User Summarize what was done: @@ -247,7 +255,7 @@ Summarize what was done: | Command | Description | | --- | --- | -| `gh issue list --label "topic:security" --label "state:agent-ready" --state open` | Find open security issues ready for fixing | +| `gh issue list --label "topic:security" --label "agent:implementation-requested" --state open` | Find security issues whose fixes a human requested | | `gh issue view --json number,title,body,state,labels,author` | Fetch full issue metadata | | `gh issue view --json comments` | Fetch all comments on an issue | | `gh pr create --title "..." --body "..."` | Create a pull request | @@ -271,11 +279,11 @@ User says: "Fix security issue #42" 8. Commit, push, and open PR with `Closes #42` 9. Report the PR link and changes to the user -### Scan and fix agent-ready issues +### Scan and fix requested security issues User says: "Fix any ready security issues" -1. Query for open issues with labels `topic:security` + `state:agent-ready` +1. Query for open issues with labels `topic:security` + `agent:implementation-requested` 2. Find issue #78: "SQL injection in search endpoint" 3. Fetch the review comment -- determination is "Legitimate concern" 4. Implement parameterized queries @@ -292,20 +300,20 @@ User says: "Fix security issue #99" 3. Report to the user: "Issue #99 was reviewed and determined to be not actionable. No fix is needed." 4. Stop -### Issue missing `state:agent-ready` label +### Directly requested issue without `agent:implementation-requested` User says: "Fix security issue #55" 1. Fetch issue #55 metadata -2. Labels are `["topic:security"]` -- missing `state:agent-ready` -3. Report to the user: "Issue #55 has the `topic:security` label but is not marked `state:agent-ready`. It may still need review or human triage before a fix can be implemented." -4. Stop +2. Labels are `["topic:security"]` -- missing `agent:implementation-requested` +3. Confirm that a legitimate security review and remediation plan exist +4. Proceed because the user's direct request authorizes implementation ### Issue without a review User says: "Fix security issue #60" -1. Fetch issue #60 metadata -- labels include both `topic:security` and `state:agent-ready` +1. Fetch issue #60 metadata -- `topic:security` is present and the user directly requested the fix 2. Fetch comments -- no `security-review-agent` comment found 3. Report to the user: "Issue #60 has not been reviewed yet. Run the review-security-issue skill first." 4. Stop diff --git a/.agents/skills/generate-sandbox-policy/SKILL.md b/.agents/skills/generate-sandbox-policy/SKILL.md index 97f6dbe31b..20e562064c 100644 --- a/.agents/skills/generate-sandbox-policy/SKILL.md +++ b/.agents/skills/generate-sandbox-policy/SKILL.md @@ -1,17 +1,17 @@ --- name: generate-sandbox-policy -description: Generate sandbox security policies from plain-language requirements and optional REST API documentation. At minimum, takes API host:port endpoints and intent to produce preset-based or L4 policies. With full API docs (OpenAPI, Swagger, markdown), generates fine-grained per-endpoint L7 rules. Trigger keywords - generate policy, create policy, update policy, change policy, sandbox policy, network policy, API policy, security policy, allow API, restrict API. +description: Generate sandbox security policies from plain-language requirements and optional REST API documentation. Produces L4 or fine-grained L7 network policies and ordered network middleware configuration. Use for API access rules, middleware host selection, failure behavior, or built-in and operator-run middleware attachment. Trigger keywords - generate policy, create policy, update policy, change policy, sandbox policy, network policy, API policy, security policy, allow API, restrict API, network middleware, supervisor middleware. --- # Generate Sandbox Policy -Generate YAML sandbox network policies from REST API documentation and natural-language user requirements. +Generate YAML sandbox network policies and network middleware configuration from API documentation and natural-language user requirements. ## Overview This skill translates a user's plain-language policy intent into a valid sandbox policy. The amount of detail the user provides determines the granularity of the generated policy — from broad L4 or preset-based policies (just a host:port) up to fine-grained per-endpoint L7 rules (full API docs). -The output is a `network_policies` YAML block (and optionally a full policy file) that conforms to the sandbox policy schema. +The output is a `network_policies` YAML block, an optional `network_middlewares` block, and optionally a full policy file that conforms to the sandbox policy schema. ## Step 1: Gather Inputs @@ -79,6 +79,7 @@ Regardless of tier, extract (or infer) these from the user's description: | **Paths** | Specific URL paths or patterns | Only for custom/fine-grained | | **Enforcement** | `enforce` or `audit`? Default to `enforce`. | No — has a default | | **Binary** | Which binary/process should have access | Yes — ask if not stated | +| **Middleware** | Whether admitted HTTP requests or client WebSocket text messages need an ordered built-in or operator-run processing stage | No | If the host and access level are clear but binaries are not specified, ask the user which binary or process will be making the requests. Suggest common defaults like `/usr/bin/curl`, `/usr/local/bin/claude`, etc. @@ -165,8 +166,15 @@ Key sections to reference: - **`L7Rule` / `L7Allow`** — method + path matching - **Access Presets** — `read-only`, `read-write`, `full` - **Private IP Access via `allowed_ips`** — CIDR allowlist for private IP space +- **Network Middleware** - top-level middleware configs, ordering, host selection, and failure behavior - **Validation Rules** — what combinations are valid/invalid +When middleware is requested, also read the full operational reference: + +``` +Read docs/extensibility/supervisor-middleware.mdx +``` + Also read the architecture overview for enforcement context. The default policy is baked into the community base image (`ghcr.io/nvidia/openshell-community/sandboxes/base:latest`). For reference, consult: ``` @@ -206,6 +214,21 @@ Is L7 inspection needed? **Critical**: `protocol: rest` on port 443 without `tls: terminate` will not work — the proxy cannot inspect encrypted traffic. Always set `tls: terminate` when combining port 443 with L7 rules. +### Middleware Decision + +Add `network_middlewares` only when the user asks to inspect, transform, redact, or independently authorize admitted HTTP requests or client WebSocket text messages. Middleware runs after network and L7 policy admission and before provider credential injection. + +- Use `openshell/regex` without gateway registration for fixed-pattern redaction of UTF-8 HTTP request bodies or complete client-to-upstream WebSocket text messages. +- Use an operator-owned middleware name only when it is already registered under `[[openshell.supervisor.middleware]]` and reachable from both the gateway and sandbox supervisors. +- Confirm that a requested WebSocket implementation exposes a `WEBSOCKET_MESSAGE/PRE_CREDENTIALS` binding. `openshell/regex` exposes this binding. A host-matched HTTP-only implementation may inspect the upgrade GET but does not join the post-upgrade chain; messages pass and OpenShell emits `binding_not_selected` coverage regardless of `on_error`. +- WebSocket middleware runs for both `ws://` and `wss://` and receives complete client text messages only. Binary messages pass under both error modes and emit `unsupported_message_type` coverage for active stages. Upstream-to-client messages remain uninspected. Do not claim that V1 provides all-message WebSocket inspection. +- Treat `fail_open` on WebSocket as a session-scoped bypass: if the stage stream fails, OpenShell disables it for later messages on that connection and emits a state-change finding. Prefer `fail_closed` for required redaction or authorization. +- `on_error` governs failures after an advertised operation binding is selected. It does not apply to an unadvertised WebSocket binding or binary-message pass-through. An explicit HTTP, WebSocket preflight, or WebSocket message denial is authoritative under both `fail_open` and `fail_closed`. +- Default `on_error` to `fail_closed`. Use `fail_open` only when bypassing the stage preserves the user's stated security requirement. +- Assign unique `order` values across the complete policy. Lower values run first, and at most 10 configs may be selected. +- Match the narrowest destination hosts possible with `endpoints.include`; use `exclude` when a broad selector has trusted exceptions. +- Do not select fail-closed middleware for `tls: skip` endpoints because the supervisor cannot inspect that traffic. + ### Mapping Paths to Glob Patterns (when building explicit rules) Only needed for the **Moderate** and **Full** tiers. Translate API path parameters to glob patterns: @@ -218,7 +241,10 @@ Only needed for the **Moderate** and **Full** tiers. Translate API path paramete | `/api/v1/models/{model_id}/versions/{version}` | `/api/v1/models/*/versions/*` | | All sub-paths under `/api/v1/` | `/api/v1/**` | -Remember: `*` does not cross `/` boundaries. Use `**` for recursive matching across path segments. +Path matching uses the runtime `glob` engine. Both `*` and `**` may cross `/` +boundaries; `?` matches one character, and bracket classes such as `[0-9]` and +`[!0]` are supported. Prefer segment-shaped patterns such as +`/repos/*/issues` for readability, but do not rely on `*` to stop at `/`. ### Building the Explicit Rules List @@ -266,6 +292,23 @@ network_policies: - { path: } ``` +When middleware is requested, add it as a separate top-level map rather than nesting it under a network policy: + +```yaml +network_middlewares: + : + name: + middleware: + order: 10 + config: {} + on_error: fail_closed + endpoints: + include: [""] + # exclude: [""] +``` + +The map key is the stable policy-local identity. Middleware selection is independent of the network policy entry that admitted the request. + ### Deny Rules Use `deny_rules` to block specific dangerous operations while allowing broad access. Deny rules are evaluated after allow rules and take precedence. This is the inverse of the `rules` approach — instead of enumerating every allowed operation, you grant broad access and block a small set of dangerous ones. @@ -337,11 +380,19 @@ Before presenting the policy to the user, verify correctness **and** flag breadt - [ ] If `tls: terminate` is set, `protocol` is also set - [ ] `rules` list is not empty when present - [ ] If `protocol: sql`, `enforcement` is not `enforce` +- [ ] Every middleware config has a non-empty `middleware` name and non-empty `endpoints.include` +- [ ] Middleware `order` values are unique and no selected chain exceeds 10 stages +- [ ] No fail-closed middleware selector can cover a `tls: skip` endpoint +- [ ] Any required WebSocket control advertises `WEBSOCKET_MESSAGE/PRE_CREDENTIALS`, and the user understands that V1 does not inspect binary messages +- [ ] Endpoints contributed by a credentialed provider are not L4-only or `tls: skip` unless `allow_uninspected_credentials: true` explicitly records the exception ### Schema Warnings (log-only, but should be fixed) - [ ] `protocol: rest` on port 443 should have `tls: terminate` - [ ] HTTP methods are standard: GET, HEAD, POST, PUT, DELETE, PATCH, OPTIONS, or `*` +- [ ] Credentialed destinations are also covered by the attached provider + profile endpoint; policy admission alone does not authorize credential + resolution ### Structural Checks @@ -349,6 +400,7 @@ Before presenting the policy to the user, verify correctness **and** flag breadt - [ ] Every endpoint has `host` and `port` - [ ] Every binary has `path` - [ ] Policy key matches `name` field +- [ ] Every middleware selector has at most 32 combined `include` and `exclude` patterns ### Breadth Warnings @@ -365,6 +417,9 @@ Evaluate the generated policy for overly broad access and **include warnings in | **Multiple broad endpoints** in one policy | "This policy grants the same broad access to N different hosts. If any of these hosts needs tighter restrictions later, you'll need to split the policy." | | **Hostless `allowed_ips`** (no `host` field) | "This endpoint has no `host` — any domain resolving to the allowed IP range on this port will be permitted. Consider adding a `host` field to restrict which domains can use this allowlist." | | **Broad CIDR** in `allowed_ips` (e.g., `10.0.0.0/8`) | "This `allowed_ips` entry covers a very broad range. Consider narrowing to a specific subnet (e.g., `10.0.5.0/24`) to minimize exposure." | +| **`on_error: fail_open`** | "This middleware can be bypassed when it is unavailable, rejects configuration, returns an invalid result, or exceeds its body limit. Use `fail_closed` unless availability is more important than this control." | +| **Broad middleware host selector** | "This middleware attaches independently of the admitting network rule to every matching destination, then runs only for operation bindings its implementation advertises. Narrow `endpoints.include` or add exclusions if the attachment is not required for every matching host." | +| **`allow_uninspected_credentials: true`** | "This endpoint may carry provider credentials on traffic OpenShell cannot inspect or rewrite. Prefer an inspected protocol and credential rewrite; keep this exception only when raw traffic is required." | Format breadth warnings clearly in the output, e.g.: @@ -397,11 +452,18 @@ The policy needs to go somewhere. Determine which mode applies: 2. **Check for conflicts**: - Does a policy with the same key already exist? If so, ask the user whether to **replace** it, **merge** new endpoints/binaries into it, or use a different key. - - Does an existing policy already cover the same host:port? Warn the user — overlapping endpoint coverage across policies causes OPA evaluation errors (complete rule conflict). + - Does an existing endpoint selector overlap the new selector? Compatible overlaps are allowed and can intentionally aggregate allow and deny rules. Reject or revise equally specific overlaps that disagree on connection or request-processing metadata, including TLS, destination constraints, protocol/parser behavior, enforcement, or credential handling. A more-specific path selector may override broader request-processing metadata. + - If the sandbox uses an attached provider credential, confirm the provider + profile also declares the intended host, port, and path. A sandbox policy + allow cannot expand the profile's static credential binding. + - For `credential_signing`, confirm an attached endpoint-bearing profile + declares `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` and covers the + signed endpoint. For an endpointless AWS profile, add + `credential_binding.provider` with the exact attached provider name. 3. **Apply the change**: - **Adding a new policy**: Insert the new policy block under `network_policies`, maintaining the file's existing indentation and style. - - **Modifying an existing policy**: Edit the specific policy in place — add/remove endpoints, change access presets, update rules, add binaries, etc. + - **Modifying an existing policy**: Edit the specific policy in place — add/remove endpoints, change access presets, update rules, add binaries, etc. A rule authorizes every binary it lists to reach every endpoint and port it lists, so adding one binary grants it all of that rule's endpoints, and adding one endpoint grants it to all of that rule's binaries. State the resulting pairs to the user before writing them. When the user wants a binary to reach only part of a rule's endpoints, put that binary and those endpoints in a separate rule instead of extending the existing one. An empty `binaries` list means any binary, so leaving it off widens the rule to every process. - **Removing a policy**: Delete the policy block if the user asks. 4. **Preserve everything else**: Do not modify `filesystem_policy`, `landlock`, `process`, or other policies unless the user explicitly asks. @@ -424,22 +486,29 @@ filesystem_policy: - /etc - /var/log read_write: - - /sandbox - /tmp - /dev/null landlock: compatibility: best_effort -process: - run_as_user: sandbox - run_as_group: sandbox - network_policies: # ``` -The `filesystem_policy`, `landlock`, and `process` sections above are sensible defaults. Tell the user these are defaults and may need adjustment for their environment. Gateway inference is configured separately through `openshell inference set/get`. The generated `network_policies` block is the primary output. +The `filesystem_policy` and `landlock` sections above are sensible defaults. +Process identity is omitted so the selected compute driver can choose it. For +Docker and Podman, each omitted identity field falls back to the image's OCI +`USER`. Tell the user these are defaults and may need adjustment for their +environment. Gateway inference is configured separately through `openshell +inference set/get`. The generated `network_policies` block is the primary +output. + +When the user explicitly requests `process.run_as_user` or +`process.run_as_group`, accept `sandbox` or a numeric UID/GID from `1` through +`4294967294`. Reject root (`0`) and the invalid identity sentinel +(`4294967295`). Warn that a low numeric identity inherits permissions granted +to the same ID on image files, mounted volumes, or devices. If the user provides a file path, write to it. Otherwise, ask where to place it. A common convention is a project-local policy file (e.g., `sandbox-policy.yaml`) passed to `openshell sandbox create --policy ` or set via the `OPENSHELL_SANDBOX_POLICY` env var. diff --git a/.agents/skills/generate-sandbox-policy/examples.md b/.agents/skills/generate-sandbox-policy/examples.md index b6acbee8bf..e6fa7ae038 100644 --- a/.agents/skills/generate-sandbox-policy/examples.md +++ b/.agents/skills/generate-sandbox-policy/examples.md @@ -727,7 +727,9 @@ An exact IP is treated as `/32` — only that specific address is permitted. **Agent workflow**: 1. Read `sandbox-policy.yaml` -2. Check that no existing policy already covers `api.github.com:443` — if one does, warn about overlap +2. Check existing selectors for `api.github.com:443`. Compatible overlaps may + aggregate request rules; revise equally specific overlaps that disagree on + TLS, destination, protocol/parser, enforcement, or credential behavior. 3. Check that the key `github_readonly` doesn't already exist 4. Insert the new policy under `network_policies`: @@ -823,17 +825,12 @@ filesystem_policy: - /etc - /var/log read_write: - - /sandbox - /tmp - /dev/null landlock: compatibility: best_effort -process: - run_as_user: sandbox - run_as_group: sandbox - network_policies: github_readonly: name: github_readonly @@ -858,7 +855,10 @@ network_policies: - { path: /usr/local/bin/claude } ``` -The agent notes that `filesystem_policy`, `landlock`, and `process` are sensible defaults that may need adjustment, and that gateway inference is configured separately via `openshell inference set/get` rather than an `inference` policy block. +The agent notes that `filesystem_policy` and `landlock` are sensible defaults +that may need adjustment. Process identity is omitted so the compute driver can +select it. Gateway inference is configured separately via `openshell inference +set/get` rather than an `inference` policy block. --- diff --git a/.agents/skills/helm-dev-environment/SKILL.md b/.agents/skills/helm-dev-environment/SKILL.md index 79c7d5bc84..2dad568c79 100644 --- a/.agents/skills/helm-dev-environment/SKILL.md +++ b/.agents/skills/helm-dev-environment/SKILL.md @@ -26,9 +26,10 @@ mise run helm:k3s:create ``` Creates a k3d cluster and merges its kubeconfig into the worktree-local `kubeconfig` file. -Also applies base manifests (`deploy/kube/manifests/agent-sandbox.yaml`) and preloads the -default community sandbox image into k3d so the first sandbox create does not wait on a -large registry pull. Traefik is disabled at cluster creation time. +Also applies the upstream agent-sandbox CRDs/controller (pinned via `AGENT_SANDBOX_VERSION` +in `tasks/scripts/helm-k3s-local.sh`, fetched from `github.com/kubernetes-sigs/agent-sandbox` +releases) and preloads the default community sandbox image into k3d so the first sandbox +create does not wait on a large registry pull. Traefik is disabled at cluster creation time. **Multi-worktree support:** the cluster name is derived from the last component of the current git branch (e.g. branch `kube-support/local-dev/tmutch` → cluster @@ -59,20 +60,39 @@ mise run helm:skaffold:dev mise run helm:skaffold:run ``` +**Supervisor sidecar topology** (build once and leave running): +```bash +mise run helm:skaffold:run:sidecar +``` + +**Supervisor sidecar topology with TLS/mTLS enabled** (build once and leave running): +```bash +mise run helm:skaffold:run:sidecar-mtls +``` + Both commands build the `gateway` and `supervisor` images and deploy the OpenShell Helm -chart. The `pkiInitJob` hook (a pre-install Job that runs `openshell-gateway generate-certs`) -generates mTLS secrets on first install. Envoy Gateway opt-in; see the Optional Add-ons section below. +chart. The sidecar profile renders an `openshell-network-init` init container for +nftables setup and an `openshell-supervisor-network` runtime sidecar for proxying. +Binary-aware policy mode runs that sidecar as UID 0 with `SYS_PTRACE` and +`DAC_READ_SEARCH`; relaxed mode can run it as the configured proxy UID, which +must be at least `1000` and distinct from the workload UID. The +sidecar-mTLS profile reuses `ci/values-sidecar.yaml` and restores +`server.disableTls=false` inline for Skaffold. The `pkiInitJob` hook (a pre-install +Job that runs `openshell-gateway generate-certs`) generates mTLS secrets on first +install. Envoy Gateway opt-in; see the Optional Add-ons section below. The gateway Service uses ClusterIP. Access is via Envoy Gateway (port `8080`) or `kubectl port-forward`. -**HA test deploy** (two gateway replicas + bundled PostgreSQL): uncomment +**HA test deploy** (two gateway replicas + external PostgreSQL Secret): uncomment `#- ci/values-high-availability.yaml` in `deploy/helm/openshell/skaffold.yaml`, -then run `mise run helm:skaffold:run` or `mise run helm:skaffold:dev`. +create the Secret named `openshell-ha-pg` with a `uri` key, then run +`mise run helm:skaffold:run` or `mise run helm:skaffold:dev`. ### TLS behaviour `ci/values-skaffold.yaml` sets `server.disableTls: true`, so Skaffold-based deploys run -plaintext by default. To test with TLS enabled, comment out that line and redeploy. +plaintext by default. To test sidecar topology with TLS enabled, use +`mise run helm:skaffold:run:sidecar-mtls`. | Mode | `server.disableTls` | Gateway scheme | |------|---------------------|----------------| @@ -124,6 +144,12 @@ openshell sandbox list --gateway-endpoint https://localhost:8090 mise run helm:skaffold:delete ``` +For a sidecar-profile deployment: + +```bash +mise run helm:skaffold:delete:sidecar +``` + ### Delete the cluster entirely ```bash @@ -176,9 +202,26 @@ To remove Keycloak: mise run keycloak:k8s:teardown ``` +### SPIRE / SPIFFE Provider Token Grants + +Skaffold can install SPIRE with the SPIFFE hardened Helm charts. To activate +SPIFFE JWT-SVIDs for dynamic provider token grants: + +1. Uncomment the `spire-crds` and `spire` releases in `deploy/helm/openshell/skaffold.yaml` +2. Uncomment `#- ci/values-spire.yaml` in the OpenShell release values files +3. Redeploy: `mise run helm:skaffold:run` + +`ci/values-spire-stack.yaml` configures the local SPIRE trust domain as +`openshell.local` and adds a `ClusterSPIFFEID` that maps sandbox pod +annotations to `spiffe://openshell.local/openshell/sandbox/`. +OpenShell mounts the SPIFFE CSI Workload API socket at +`/spiffe-workload-api/spire-agent.sock` into sandbox pods for provider token +grants. Supervisor-to-gateway authentication remains on the Kubernetes +ServiceAccount bootstrap and gateway-minted sandbox JWT path. + --- -## Cluster Lifecycle (suspend/resume) +## Cluster Lifecycle (stop/start) Stop the cluster without losing state (faster than delete/recreate): ```bash @@ -193,6 +236,33 @@ mise run helm:k3s:status --- +## Helm Chart Checks + +Run the chart lint task before changing Helm templates, values overlays, or +Skaffold inputs: + +```bash +mise run helm:lint +``` + +If Helm reports missing chart dependencies, remove the specific stale subchart +archive or directory named by the error from `deploy/helm/openshell/charts/`, +then rerun the lint task. + +For example, when lint reports `chart metadata is missing these dependencies: +postgresql`, remove stale PostgreSQL chart artifacts: + +```bash +rm -f deploy/helm/openshell/charts/postgresql-*.tgz +rm -rf deploy/helm/openshell/charts/postgresql +mise run helm:lint +``` + +The `charts/` directory is ignored and regenerated by `helm dependency build` +for dependencies still declared in `Chart.yaml`. + +--- + ## Key Files | Path | Purpose | @@ -202,8 +272,11 @@ mise run helm:k3s:status | `deploy/helm/openshell/ci/values-skaffold.yaml` | Dev overrides (image pull policy, TLS disabled for local Skaffold) | | `deploy/helm/openshell/ci/values-cert-manager.yaml` | cert-manager PKI overlay (opt-in; disables pkiInitJob) | | `deploy/helm/openshell/ci/values-gateway.yaml` | Envoy Gateway GRPCRoute + Gateway overlay | -| `deploy/helm/openshell/ci/values-high-availability.yaml` | HA test overlay (`replicaCount: 2` with bundled PostgreSQL) | +| `deploy/helm/openshell/ci/values-high-availability.yaml` | HA test overlay (`replicaCount: 2` with external PostgreSQL Secret) | | `deploy/helm/openshell/ci/values-keycloak.yaml` | Keycloak OIDC overlay | +| `deploy/helm/openshell/ci/values-sidecar.yaml` | Supervisor sidecar topology overlay for Kubernetes e2e/dev | +| `deploy/helm/openshell/ci/values-spire.yaml` | SPIFFE/SPIRE provider token grant overlay | +| `deploy/helm/openshell/ci/values-spire-stack.yaml` | SPIRE hardened chart values for local dev | | `deploy/helm/openshell/ci/values-tls-disabled.yaml` | Lint-only: TLS + auth disabled (reverse-proxy edge termination) | | `deploy/kube/manifests/envoy-gateway-openshell.yaml` | GatewayClass for Envoy Gateway (`mise run helm:gateway:apply`) | | `tasks/scripts/helm-k3s-local.sh` | k3d cluster create/delete/start/stop/status | diff --git a/.agents/skills/launch-openshell-gator/SKILL.md b/.agents/skills/launch-openshell-gator/SKILL.md new file mode 100644 index 0000000000..8b25760698 --- /dev/null +++ b/.agents/skills/launch-openshell-gator/SKILL.md @@ -0,0 +1,406 @@ +--- +name: launch-openshell-gator +description: Launch and supervise OpenShell gator agents. Use when starting gator on issues or PRs, checking gator sandboxes, building the gator sandbox image, restarting stuck gators, inspecting gator logs, or experimenting with gator harness/model overrides. Trigger keywords - launch gator, start gator, run gator, gator sandbox, supervised gator, gator logs, restart gator. +--- + +# Launch OpenShell Gator + +Launch and supervise the repository's headless gator sandbox agent through OpenShell. This skill covers the operator workflow around `scripts/agents/run.sh`; the in-sandbox review and state-machine policy remains the `gator-gate` skill baked into the gator payload. + +For gator's PR/issue validation policy, load `gator-gate` inside the launched sandbox. For generic sandbox CLI usage, use `openshell-cli`. For unhealthy gateways or sandbox startup failures, use `debug-openshell-cluster` after the launch preflight identifies a gateway/runtime problem. + +## Non-Negotiable Rules + +- Keep normal gator launches supervised: use `--watch --background` and let the in-sandbox supervisor own sleeping and relaunching bounded cycles. +- Do not add passive `sleep` loops in the operator session to watch gator. Check logs or status once, then report the current state or launch a proper watcher outside the model session only when explicitly asked. +- Do not change the default gator model in `scripts/agents/gator/agent.yaml` for experiments. Use `CODEX_MODEL=...` and, if needed, a temporary `--from` Docker context or `--codex-bin` override. +- Do not push to contributor branches, approve, merge, post `/ok to test`, or broaden gator scope unless the operator explicitly authorized that action. +- Scope each launch prompt to the requested issue/PR set. Avoid repo-wide gator scans unless the operator asked for repo-wide processing. +- Leave unrelated local files alone, including `.opencode/` artifacts and old gator logs unless the user asks for cleanup. + +## Key Paths + +| Path | Purpose | +|---|---| +| `scripts/agents/run.sh` | Manifest-driven OpenShell agent launcher. | +| `scripts/agents/gator/agent.yaml` | Gator manifest: immutable payload version, default gateway, harness, providers, runtime, skills, and subagents. | +| `scripts/agents/gator/Dockerfile` | Gator sandbox image source. Local launches build this image through OpenShell. | +| `scripts/agents/gator/policy.yaml` | Sandbox policy for the gator agent. | +| `scripts/agents/gator/bin/gh` | Gator-specific `gh` wrapper and same-SHA duplicate-post guard. | +| `scripts/agents/gator/bin/review-feedback-ledger` | Builds tree-aware review scope, durable findings, convergence telemetry, and review-budget state. | +| `scripts/agents/gator/bin/validate-review-findings` | Enforces the blocker evidence schema and downgrades unsupported hypotheses. | +| `scripts/agents/gator/prompts/gator.md` | Rendered top-level prompt template baked into the payload. | +| `scripts/agents/gator/skills/gator-gate/SKILL.md` | In-sandbox gator state-machine skill. | +| `scripts/agents/gator/logs/` | Background launch and supervisor logs. | + +## Preflight + +Run these checks before launching unless the operator asks for a best-effort launch. + +### Step 1: Confirm Repository Root + +```bash +git rev-parse --show-toplevel +git status --short --branch +``` + +Use the repository root as the working directory for all commands. A dirty worktree is allowed, but do not stage or modify unrelated files. + +### Step 2: Verify Required Host Tools + +```bash +command -v openshell +command -v gh +command -v jq +command -v ruby +``` + +The local `openshell` wrapper may recompile the CLI. If that fails, fix the local build or ask the operator before changing unrelated source. + +### Step 3: Verify GitHub Auth + +Use `gh api user` as the health check. It works with provider-scoped tokens and matches gator's own auth guidance. + +```bash +gh api user --jq '.login' +gh api repos/NVIDIA/OpenShell --jq '{full_name,default_branch}' +``` + +If this fails, refresh host `gh` auth before launching. Do not rely on `gh auth status` alone inside provider-backed sandboxes. + +### Step 4: Verify Codex Auth For Codex Harness + +The default gator harness is Codex. Check that the host has usable Codex auth material: + +```bash +jq -e '.tokens.access_token and .tokens.refresh_token and .tokens.account_id' "$HOME/.codex/auth.json" >/dev/null +``` + +If this fails, run the local Codex login flow outside the gator launch. If Codex was recently reauthenticated and gateway refresh fails later, relaunch with `--reset-refresh` once. + +### Step 5: Verify Gateway Is Registered And Alive + +Use the target gateway from the operator request or current session context. Do not assume a gateway name. If the operator did not specify one, list registered gateways and ask before launching when the correct target is ambiguous. + +```bash +openshell gateway list + +gateway_name="" +[[ "$gateway_name" =~ ^[A-Za-z0-9_.-]+$ ]] || { echo "invalid gateway name" >&2; exit 1; } + +openshell --gateway "$gateway_name" status +openshell --gateway "$gateway_name" sandbox list +``` + +Expected result: status returns successfully and sandbox listing completes. If the gateway is unreachable, the runtime cannot create sandboxes, or sandbox listing hangs, switch to `debug-openshell-cluster` and fix the gateway before launching gator. + +### Step 6: Check Existing Gator Sandboxes + +Avoid duplicate gators for the same PR unless intentionally replacing a stuck or stale one. + +```bash +gateway_name="" +[[ "$gateway_name" =~ ^[A-Za-z0-9_.-]+$ ]] || { echo "invalid gateway name" >&2; exit 1; } + +openshell --gateway "$gateway_name" sandbox list +``` + +Look for names like `gator-pr--supervised`. If one exists, inspect its log before deleting or relaunching. + +## Input Normalization + +Never paste raw operator text into shell arguments such as `--gateway`, `--name`, `--from`, issue numbers, or PR numbers. Normalize values before constructing launch commands. + +Use the operator-specified gateway or a gateway selected from `openshell gateway list`: + +```bash +gateway_name="" +[[ "$gateway_name" =~ ^[A-Za-z0-9_.-]+$ ]] || { echo "invalid gateway name" >&2; exit 1; } +``` + +Use digits only for issue and PR numbers: + +```bash +pr_number="" +[[ "$pr_number" =~ ^[0-9]+$ ]] || { echo "invalid PR number" >&2; exit 1; } +``` + +Use the portable Kubernetes DNS-1123 sandbox-name format even when the selected gateway currently uses another driver: + +```bash +sandbox_name="gator-pr-${pr_number}-supervised" +[[ "$sandbox_name" =~ ^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$ ]] || { echo "invalid sandbox name" >&2; exit 1; } +``` + +For local image contexts passed to `--from`, use an agent-created path such as `mktemp -d`; do not pass raw user-supplied paths without validating that they are expected local Dockerfile contexts. + +## Standard Launches + +### Launch A PR Watcher + +Use a stable, scoped name and a prompt that names exactly what gator should do. + +```bash +gateway_name="" +pr_number="" +[[ "$gateway_name" =~ ^[A-Za-z0-9_.-]+$ ]] || { echo "invalid gateway name" >&2; exit 1; } +[[ "$pr_number" =~ ^[0-9]+$ ]] || { echo "invalid PR number" >&2; exit 1; } +sandbox_name="gator-pr-${pr_number}-supervised" +[[ "$sandbox_name" =~ ^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$ ]] || { echo "invalid sandbox name" >&2; exit 1; } + +./scripts/agents/run.sh \ + --agent gator \ + --gateway "$gateway_name" \ + --name "$sandbox_name" \ + --watch \ + --background \ + "Review and monitor PR #${pr_number} through the gator-gate workflow. Scope this invocation only to PR #${pr_number}." +``` + +The launcher builds the gator sandbox image when needed, stages the immutable payload, imports provider profiles, configures provider credentials and refresh, creates the sandbox, and writes a background log under `scripts/agents/gator/logs/`. + +### Launch An Issue Or Issue/PR Pair + +```bash +gateway_name="" +issue_number="" +[[ "$gateway_name" =~ ^[A-Za-z0-9_.-]+$ ]] || { echo "invalid gateway name" >&2; exit 1; } +[[ "$issue_number" =~ ^[0-9]+$ ]] || { echo "invalid issue number" >&2; exit 1; } +sandbox_name="gator-issue-${issue_number}-supervised" +[[ "$sandbox_name" =~ ^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$ ]] || { echo "invalid sandbox name" >&2; exit 1; } + +./scripts/agents/run.sh \ + --agent gator \ + --gateway "$gateway_name" \ + --name "$sandbox_name" \ + --watch \ + --background \ + "Run gator on issue #${issue_number}. Scope this invocation only to issue #${issue_number}." +``` + +For a linked pair: + +```bash +gateway_name="" +pr_number="" +issue_number="" +[[ "$gateway_name" =~ ^[A-Za-z0-9_.-]+$ ]] || { echo "invalid gateway name" >&2; exit 1; } +[[ "$pr_number" =~ ^[0-9]+$ ]] || { echo "invalid PR number" >&2; exit 1; } +[[ "$issue_number" =~ ^[0-9]+$ ]] || { echo "invalid issue number" >&2; exit 1; } +sandbox_name="gator-pr-${pr_number}-supervised" +[[ "$sandbox_name" =~ ^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$ ]] || { echo "invalid sandbox name" >&2; exit 1; } + +./scripts/agents/run.sh \ + --agent gator \ + --gateway "$gateway_name" \ + --name "$sandbox_name" \ + --watch \ + --background \ + "Review and monitor PR #${pr_number} with linked issue #${issue_number} through the gator-gate workflow. Scope this invocation only to PR #${pr_number} and issue #${issue_number}." +``` + +### Launch With Explicit Maintainer Authorization + +Only include authorization in the prompt when the operator explicitly gave it. + +```bash +gateway_name="" +pr_number="" +[[ "$gateway_name" =~ ^[A-Za-z0-9_.-]+$ ]] || { echo "invalid gateway name" >&2; exit 1; } +[[ "$pr_number" =~ ^[0-9]+$ ]] || { echo "invalid PR number" >&2; exit 1; } +sandbox_name="gator-pr-${pr_number}-supervised" +[[ "$sandbox_name" =~ ^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$ ]] || { echo "invalid sandbox name" >&2; exit 1; } + +./scripts/agents/run.sh \ + --agent gator \ + --gateway "$gateway_name" \ + --name "$sandbox_name" \ + --watch \ + --background \ + "Review and monitor PR #${pr_number} through the gator-gate workflow. Scope this invocation only to PR #${pr_number}. The operator explicitly authorizes applying the test:e2e label, posting /ok to test for the current head SHA, and rerunning the relevant current-head workflow when the E2E Label Help bot says that is required." +``` + +## Model Or Image Experiments + +Use environment overrides. Do not edit `agent.yaml` for temporary experiments. + +```bash +gateway_name="" +pr_number="" +[[ "$gateway_name" =~ ^[A-Za-z0-9_.-]+$ ]] || { echo "invalid gateway name" >&2; exit 1; } +[[ "$pr_number" =~ ^[0-9]+$ ]] || { echo "invalid PR number" >&2; exit 1; } +sandbox_name="gator-pr-${pr_number}-gpt56sol-supervised" +[[ "$sandbox_name" =~ ^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$ ]] || { echo "invalid sandbox name" >&2; exit 1; } + +CODEX_MODEL=gpt-5.6-sol \ +./scripts/agents/run.sh \ + --agent gator \ + --gateway "$gateway_name" \ + --name "$sandbox_name" \ + --watch \ + --background \ + "Review and monitor PR #${pr_number} through the gator-gate workflow. Scope this invocation only to PR #${pr_number}. This launch is intentionally testing Codex model gpt-5.6-sol via the CLI launcher." +``` + +If the installed Codex CLI is too old for a model, create a temporary copy of `scripts/agents/gator/`, adjust only that temporary Dockerfile, and launch with that generated context. Keep the repo Dockerfile unchanged unless the version bump is the intended code change. + +Example shape: + +```bash +gateway_name="" +pr_number="" +[[ "$gateway_name" =~ ^[A-Za-z0-9_.-]+$ ]] || { echo "invalid gateway name" >&2; exit 1; } +[[ "$pr_number" =~ ^[0-9]+$ ]] || { echo "invalid PR number" >&2; exit 1; } +sandbox_name="gator-pr-${pr_number}-gpt56sol-supervised" +[[ "$sandbox_name" =~ ^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$ ]] || { echo "invalid sandbox name" >&2; exit 1; } +tmp_context="$(mktemp -d "${TMPDIR:-/tmp}/gator-codex-XXXXXX")" +cp -R scripts/agents/gator/. "$tmp_context"/ + +CODEX_MODEL=gpt-5.6-sol \ +./scripts/agents/run.sh \ + --agent gator \ + --gateway "$gateway_name" \ + --name "$sandbox_name" \ + --from "$tmp_context" \ + --watch \ + --background \ + "Review and monitor PR #${pr_number} through the gator-gate workflow. Scope this invocation only to PR #${pr_number}." +``` + +## Monitoring + +### Read The Launch Result + +The launcher prints the log path when `--background` is used: + +```text +Started in background. Log: scripts/agents/gator/logs/.log +``` + +Read that file directly. Important markers: + +- `Built image ...` means the local image build completed. +- `Created sandbox: ` means OpenShell accepted the sandbox. +- `openshell-agent: starting watch cycle` means the in-sandbox supervisor began a bounded cycle. +- `OpenAI Codex v...` plus `model: ...` confirms the Codex CLI and model actually used. +- `OPENSHELL_AGENT_RESULT {...}` is the bounded-cycle sentinel. In watch mode, the supervisor sleeps and relaunches after this line. +- `openshell-agent: still running watch cycle ...` is a heartbeat during long active model cycles. +- `review_feedback_lookup_failed` means Gator could not build the required cross-SHA feedback ledger and deliberately skipped a context-free review. + +### Inspect Active Sandboxes + +```bash +gateway_name="" +sandbox_name="" +[[ "$gateway_name" =~ ^[A-Za-z0-9_.-]+$ ]] || { echo "invalid gateway name" >&2; exit 1; } +[[ "$sandbox_name" =~ ^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$ ]] || { echo "invalid sandbox name" >&2; exit 1; } + +openshell --gateway "$gateway_name" sandbox list +openshell --gateway "$gateway_name" sandbox get "$sandbox_name" +``` + +If `sandbox get` is not supported by the local CLI shape, use `openshell sandbox --help` and follow the current command help. + +### Interpret Common Sentinels + +| Sentinel | Meaning | Operator action | +|---|---|---| +| `status=waiting` | Normal watch wait. | Leave sandbox running. | +| `status=blocked` | Human/process blocker. | Read reason; decide whether a human action is needed. | +| `status=transient_failure` | Retryable infrastructure/auth/transport issue. | Let supervisor retry unless repeated failures hit the configured cap. | +| `status=terminal_failure` | Unrecoverable or stale immutable payload. | Inspect the reason; rebuild/relaunch for `stale_gator_payload`. | +| `status=complete` | Target closed, merged, or one-shot complete. | Delete sandbox if no longer needed. | + +## Restarting A Gator + +Restart when the payload must change, the sandbox is wedged without a sentinel, the model/tooling version changed, or a transient failure repeats past the useful retry point. + +Increment `payload_version` in `scripts/agents/gator/agent.yaml` whenever a +merged change alters the Gator prompt, gate skill, reviewer contract, write +guard, ledger, or bundled validator. Existing immutable watchers cannot replace +their own payload. New-version watchers detect later published versions and +stop with `stale_gator_payload`; relaunch every still-active older watcher after +the version bump is published. + +Before deleting, check that the sandbox is truly stale or that the operator asked for a restart. If a bounded review cycle is actively running and still producing useful output, prefer leaving it alone. + +```bash +gateway_name="" +sandbox_name="" +[[ "$gateway_name" =~ ^[A-Za-z0-9_.-]+$ ]] || { echo "invalid gateway name" >&2; exit 1; } +[[ "$sandbox_name" =~ ^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$ ]] || { echo "invalid sandbox name" >&2; exit 1; } + +openshell --gateway "$gateway_name" sandbox delete "$sandbox_name" +./scripts/agents/run.sh \ + --agent gator \ + --gateway "$gateway_name" \ + --name "$sandbox_name" \ + --watch \ + --background \ + "" +``` + +When relaunching after a same-SHA infrastructure failure, say that the prior attempt failed before producing a valid review disposition. When relaunching after a draft-only blocker cleared, say that the prior same-SHA disposition was only a draft blocker and the PR is now ready for review. + +## Troubleshooting + +### Gateway Unreachable + +Symptoms: `openshell status` fails, `sandbox list` fails, sandbox remains pending, image build never starts. + +Action: load `debug-openshell-cluster` and diagnose the gateway/driver. Do not keep retrying gator launches against a dead gateway. + +### Image Build Failure + +Symptoms: Dockerfile step failure, missing package, incompatible Codex CLI, registry pull failure. + +Actions: + +- Confirm the build context is `scripts/agents/gator/` or the intended temporary `--from` context. +- Confirm Docker or the selected gateway runtime can pull `nvcr.io/nvidia/base/ubuntu:noble-20251013`. +- For Codex CLI version experiments, adjust a temporary Docker context first. +- Do not commit Dockerfile version changes unless the repo should permanently use that version. + +### Provider Or Credential Failure + +Symptoms: host `gh` auth fails, Codex refresh fails, in-sandbox GitHub calls report auth failures, `reviewer_subagent_failed` repeats due Codex auth. + +Actions: + +- Re-run the GitHub and Codex preflight checks. +- If host Codex auth changed, relaunch with `--reset-refresh` once. +- If Entra or Microsoft auth is involved in a future provider, use the relevant auth skill. Gator's default providers are GitHub and Codex. + +### Unsupported `gh pr view --json` Field + +Gator may recover by using supported `gh pr view` fields plus REST calls. If it does not, patch the gator prompt or skill to avoid the unsupported field, validate, commit, and relaunch with the updated payload. + +### Same-SHA Duplicate Guard Blocks A Needed Comment + +The wrapper intentionally blocks duplicate same-head-SHA gator dispositions. A relaunch should not post again for the same SHA unless one of these applies: + +- Maintainer explicitly requests a same-SHA public response. +- The PR is merged or closed and needs terminal cleanup. +- The earlier attempt failed before posting. +- The prior marked disposition was only a reviewer infrastructure failure. +- The prior marked disposition was only a draft blocker and the PR is now ready for review. +- A state-specific TTL nudge is due after 48 business hours. The nudge may request + the pending human action, but it must not repeat the review disposition or + trigger another reviewer run. + +Do not bypass with `OPENSHELL_GATOR_ALLOW_SAME_SHA_COMMENT=1` unless the operator explicitly confirms a maintainer override. + +## Reporting Back + +When you launch or inspect gator, report: + +- Sandbox name. +- Gateway name. +- Log path. +- Target issue/PR scope. +- Harness and model when relevant. +- Whether image build and sandbox creation succeeded. +- Latest sentinel or heartbeat status. +- Any human action needed. + +Keep the report concise. Include exact commands only when they help the operator reproduce or continue the workflow. diff --git a/.agents/skills/openshell-cli/SKILL.md b/.agents/skills/openshell-cli/SKILL.md index 4b7501c1fe..8aae80fd0a 100644 --- a/.agents/skills/openshell-cli/SKILL.md +++ b/.agents/skills/openshell-cli/SKILL.md @@ -1,6 +1,6 @@ --- name: openshell-cli -description: Guide agents through using the OpenShell CLI (openshell) for sandbox management, gateway registration, provider configuration, policy iteration, BYOC workflows, and inference routing. Covers basic through advanced multi-step workflows. Trigger keywords - openshell, sandbox create, sandbox connect, logs, provider create, policy set, policy get, image push, forward, port forward, BYOC, bring your own container, use openshell, run openshell, CLI usage, manage sandbox, manage provider, gateway add, gateway select. +description: Guide agents through using the OpenShell CLI (openshell) for sandbox management, gateway registration, provider configuration and refresh, policy iteration, settings, service exposure, BYOC workflows, and inference routing. Covers basic through advanced multi-step workflows. Trigger keywords - openshell, sandbox create, sandbox exec, sandbox connect, logs, provider create, provider profile, provider refresh, policy set, policy get, settings, service expose, forward, port forward, BYOC, bring your own container, inference, use openshell, run openshell, CLI usage, manage sandbox, manage provider, gateway add, gateway select. --- # OpenShell CLI @@ -9,7 +9,7 @@ Guide agents through using the `openshell` CLI for sandbox and platform manageme ## Overview -The OpenShell CLI (`openshell`) is the primary interface for managing sandboxes, providers, policies, inference routes, and gateway registrations. Gateway service lifecycle is handled outside the CLI by packages, systemd, Helm, or development tasks. This skill teaches agents how to orchestrate CLI commands for common and complex workflows. +The OpenShell CLI (`openshell`) is the primary interface for managing sandboxes, providers, policies, settings, exposed services, inference routes, and gateway registrations. Gateway service lifecycle is handled outside the CLI by packages, systemd, Helm, or development tasks. This skill teaches agents how to orchestrate CLI commands for common and complex workflows. **Companion skill**: For creating or modifying sandbox policy YAML content (network rules, L7 inspection, access presets), use the `generate-sandbox-policy` skill. This skill covers the CLI *commands* for the policy lifecycle; `generate-sandbox-policy` covers policy *content authoring*. @@ -32,7 +32,7 @@ This is your primary fallback. Use it freely -- the CLI's help output is authori ## Command Reference -See [cli-reference.md](cli-reference.md) for the full command tree with all flags and options. Use it as a quick-reference to avoid round-tripping through `--help` for common commands. +See [cli-reference.md](cli-reference.md) for the current command tree and commonly used flags. Use it as a quick-reference, then confirm uncommon or security-sensitive options with `--help`. --- @@ -52,9 +52,15 @@ Use an `http://` endpoint only for trusted local port-forwarding or a protected ```bash openshell status +openshell whoami ``` -Confirm the gateway is reachable and shows a version. +Confirm the gateway is reachable, authentication is valid or not required, and +the output shows a version. `Status: Connected` only proves the public health +endpoint is reachable; inspect the separate `Authentication` line before +running protected commands. `openshell whoami` reports the identity validated +by the gateway, including the subject an administrator uses for workspace +membership. Add `--output json` for automation. ### Step 3: Create a sandbox @@ -66,6 +72,8 @@ openshell sandbox create This creates a sandbox with defaults and drops you into an interactive shell. +When supplying `--name`, use a portable DNS-1123 label: at most 63 lowercase alphanumeric or `-` characters, beginning and ending with an alphanumeric character. The Kubernetes driver rejects uppercase letters, underscores, dots, and other names that cannot become Kubernetes resource labels. + **Shortcut for known tools**: When the trailing command is a recognized tool, the CLI auto-creates the required provider from local credentials: ```bash @@ -87,9 +95,12 @@ openshell sandbox delete ## Workflow 2: Provider Management -Providers supply credentials to sandboxes (API keys, tokens, etc.). Manage them before creating sandboxes that need them. +Providers supply credentials and provider-specific configuration to sandboxes. Provider types come from built-in and custom profiles; do not rely on a hard-coded type list. Discover the profiles available on the selected gateway: -Supported types: `claude`, `opencode`, `codex`, `generic`, `nvidia`, `gitlab`, `github`, `outlook`. +```bash +openshell provider list-profiles +openshell provider list-profiles --output json +``` ### Create a provider from local credentials @@ -103,25 +114,97 @@ The `--from-existing` flag discovers credentials from local state (e.g., `gh aut ```bash openshell provider create --name my-api --type generic \ - --credential API_KEY=sk-abc123 \ + --credential API_KEY \ --config base_url=https://api.example.com ``` -Bare `KEY` (without `=VALUE`) reads the value from the environment variable of that name: +Bare `KEY` reads the value from the environment variable of that name and avoids placing the secret in shell history. Use `KEY=VALUE` only when the user explicitly accepts that exposure. + +Other credential sources are `--from-gcloud-adc` for compatible profiles and `--runtime-credentials` when the gateway or sandbox resolves the required credentials at runtime. + +Static provider credentials resolve only for hosts, ports, and paths declared by +the provider profile. Use `provider profile export` to inspect that boundary +when a placeholder is present but requests receive +`credential_endpoint_mismatch`. A profileless static provider fails closed +because the gateway cannot construct a binding. + +When an inspected request receives `request_authority_mismatch`, compare its +HTTP authority with the CONNECT tunnel endpoint. The host and effective port +must match. For a tunnel to `api.example.com:8443`, send +`Host: api.example.com:8443`; `Host: api.example.com` omits the non-default +port and is rejected. An absolute-form request target must use the same +authority. + +Profile-backed provider policy composition is controlled by the gateway-global +`providers_v2_enabled` setting. Static credential endpoint binding remains +active even when policy composition is disabled: ```bash -openshell provider create --name my-api --type generic --credential API_KEY +openshell settings get --global +openshell settings set --global --key providers_v2_enabled --value true +``` + +### Inspect and manage provider profiles + +```bash +openshell provider profile export github --output yaml +openshell provider profile lint --file ./my-profile.yaml +openshell provider profile import --file ./my-profile.yaml ``` ### List, inspect, update, delete ```bash openshell provider list +openshell provider list --output json openshell provider get my-github -openshell provider update my-github --type github --from-existing +openshell provider update my-github --from-existing openshell provider delete my-github ``` +`provider update` does not take `--type`. It updates credentials, config, or credential expiry on the existing provider. + +### Configure credential refresh + +Use refresh commands only when the provider profile and gateway support refreshable credentials: + +```bash +openshell provider refresh status my-outlook +openshell provider refresh configure my-outlook \ + --credential-key MS_GRAPH_ACCESS_TOKEN \ + --strategy oauth2-refresh-token \ + --secret-material-env REFRESH_TOKEN=MS_GRAPH_REFRESH_TOKEN \ + --credential-expires-at 2026-07-16T00:00:00Z +openshell provider refresh rotate my-outlook --credential-key MS_GRAPH_ACCESS_TOKEN +``` + +Prefer `--secret-material-env KEY[=ENVVAR]` for secret refresh material. `--material KEY=VALUE` is for non-secret material; `--secret-material-key` marks supplied material keys as secret. + +The gateway stores secret refresh material through its active credential driver. +With Vault selected, refresh tokens, client secrets, and private keys live in +Vault alongside injectable provider credentials; refresh state contains only +opaque handles. A credential-backend read or write failure makes refresh fail +closed rather than falling back to inline storage. Before OpenShell 0.1.0, the +gateway does not migrate legacy inline refresh material or move secrets between +credential backends. Reconfigure affected grants after upgrading, and remove or +reconfigure credentials while the original backend remains available before +changing backends. Do not run mixed gateway versions against the same refresh +records. + +Gateway-managed refresh credentials use an identity-stable workload handle. +Routine automatic refresh and `provider refresh rotate` update the access token +behind that handle, so long-running processes do not need to restart. Running +processes must be restarted once when upgrading from revision-scoped +placeholders. A later `provider refresh configure` call is an explicit +reauthorization boundary: it revokes the previous handle, and processes holding +that handle fail closed until restarted. + +While gateway-managed refresh is configured, `provider update --credential` +cannot replace or delete the refresh-owned primary credential or any co-minted +output. Use `provider refresh rotate`, reconfigure refresh, or delete refresh +before returning those keys to manual management. Unrelated provider fields +remain updateable. + --- ## Workflow 3: Sandbox Lifecycle @@ -134,29 +217,41 @@ openshell sandbox create \ --provider my-github \ --provider my-claude \ --policy ./my-policy.yaml \ - --upload .:/sandbox \ + --upload .:/workspace \ + --label team=agents \ -- claude ``` Key flags: -- `--provider`: Attach one or more providers (repeatable) +- `--provider`: Attach configured credential providers for API keys, tokens, and other secrets (repeatable) - `--policy`: Custom policy YAML (otherwise uses built-in default or `OPENSHELL_SANDBOX_POLICY` env var) +- `--gpu [COUNT]`: Request the driver's default GPU selection or a specific GPU count - `--cpu`, `--memory`: Set per-sandbox compute sizing. Docker/Podman apply limits; Kubernetes applies matching requests and limits. -- `--upload [:]`: Upload local files into the sandbox (default dest: `/sandbox`) +- `--driver-config-json`: Pass experimental driver-specific sandbox configuration +- `--label KEY=VALUE`: Add labels for later selection (repeatable) +- `--env KEY=VALUE`: Set non-secret sandbox environment variables (repeatable); use `--provider` for credentials +- `--approval-mode manual|auto`: Control handling of agent-authored policy proposals; `manual` is the default +- `--upload [:]`: Upload local files into the container working directory or an explicit destination +- `--no-git-ignore`: Disable `.gitignore` filtering for uploads - `--no-keep`: Delete the sandbox after the initial command or shell exits -- `--forward `: Forward a local port and keep the sandbox alive +- `--forward [BIND_ADDRESS:]PORT`: Forward a local port and keep the sandbox alive +- `--editor vscode|cursor`: Open a remote editor after creation and keep the sandbox alive ### List and inspect sandboxes ```bash openshell sandbox list +openshell sandbox list --selector team=agents --output json openshell sandbox get my-sandbox ``` +Most commands with an optional sandbox name use the last-used sandbox. Pass an explicit name in automation. + ### Connect to a running sandbox ```bash openshell sandbox connect my-sandbox +openshell sandbox connect my-sandbox --editor vscode ``` Opens an interactive SSH shell. To configure VS Code Remote-SSH: @@ -168,11 +263,40 @@ openshell sandbox ssh-config my-sandbox >> ~/.ssh/config ### Upload and download files ```bash -# Upload local files to sandbox -openshell sandbox upload my-sandbox ./src /sandbox/src +# Upload local files to the sandbox working directory +openshell sandbox upload my-sandbox ./src -# Download files from sandbox -openshell sandbox download my-sandbox /sandbox/output ./local-output +# Download a path relative to the sandbox working directory +openshell sandbox download my-sandbox output ./local-output +``` + +Uploads honor `.gitignore` by default. Add `--no-git-ignore` only when ignored files are intentionally in scope. + +Uploads preserve symlinks, including dangling symlinks, instead of dereferencing their targets. A symlink source bypasses Git-aware filtering so the link itself is archived. + +When the upload destination is omitted, the CLI discovers the remote working +directory. Uploading a named directory merges it into an existing directory of +the same name, overwriting matching entries without deleting unrelated entries. +Downloads accept paths relative to that working directory or absolute paths +within it. + +### Execute a non-interactive command + +```bash +openshell sandbox exec --name my-sandbox --workdir /workspace -- ls -la +openshell sandbox exec --name my-sandbox --env MODE=test -- cargo test +``` + +`sandbox exec` streams output and exits with the remote command's exit code. Use `sandbox connect` for an interactive shell. +Use `--env` only for non-secret values. Attach credentials to the sandbox with a +provider instead of passing API keys, tokens, or other secrets to `sandbox exec`. + +### Change attached providers + +```bash +openshell sandbox provider list my-sandbox +openshell sandbox provider attach my-sandbox my-github +openshell sandbox provider detach my-sandbox my-github ``` ### View logs @@ -196,15 +320,31 @@ openshell logs my-sandbox --since 5m ```bash openshell sandbox delete my-sandbox openshell sandbox delete sandbox-1 sandbox-2 sandbox-3 # Multiple at once +openshell sandbox delete --all +``` + +### Stop and start sandboxes + +Use stop to halt compute while retaining the sandbox and its persistent +workspace: + +```bash +openshell sandbox stop [name] +openshell sandbox start [name] ``` +Both commands default to the last-used sandbox. Stop stops background +forwards and waits for `Stopped`; start waits for `Ready`. Connect, exec, +file transfer, forwarding, and exposed services are unavailable while +stopped. Delete remains the operation that removes retained state. + --- ## Workflow 4: Policy Iteration Loop This is the most important multi-step workflow. It enables a tight feedback cycle where sandbox policy is refined based on observed activity. -**Key concept**: Policies have static fields (immutable after creation: `filesystem_policy`, `landlock`, `process`) and one dynamic field (`network_policies`). Only `network_policies` can be updated without recreating the sandbox. +**Key concept**: Policies have static fields (immutable after creation: `filesystem_policy`, `landlock`, `process`) and two dynamic fields: `network_policies` and `network_middlewares`. Both dynamic fields can be updated without recreating the sandbox. ``` Create sandbox with initial policy @@ -256,19 +396,22 @@ Look for log lines with `action: deny` -- these indicate blocked network request openshell policy get dev --full > current-policy.yaml ``` -The `--full` flag outputs valid YAML that can be directly re-submitted. This is the round-trip format. +The `--full` flag includes the effective policy, including provider-composed entries. Use `--base` instead when the editable base policy is needed without provider-composed entries. Before resubmitting a `--full` result, review composed entries and prefer incremental updates or the base policy when appropriate. ### Step 4: Modify the policy Edit `current-policy.yaml` to allow the blocked actions. **For policy content authoring, delegate to the `generate-sandbox-policy` skill.** That skill handles: - Network endpoint rule structure -- L4 vs L7 policy decisions +- L4 vs REST, WebSocket, JSON-RPC, MCP, and SQL L7 policy decisions - Access presets (`read-only`, `read-write`, `full`) - TLS termination configuration - Enforcement modes (`audit` vs `enforce`) - Binary matching patterns +- Ordered `network_middlewares`, host selection, HTTP and WebSocket bindings, and `fail_open` or `fail_closed` behavior + +`network_policies` and `network_middlewares` can be modified at runtime. If `filesystem_policy`, `landlock`, or `process` need changes, the sandbox must be recreated. Built-in middleware such as `openshell/regex` needs no gateway registration. An operator-run middleware must already be registered under `[[openshell.supervisor.middleware]]`; changing that static registration requires a gateway restart. -Only `network_policies` can be modified at runtime. If `filesystem_policy`, `landlock`, or `process` need changes, the sandbox must be recreated. +Middleware can inspect parsed HTTP request bodies and complete client-to-upstream WebSocket text messages over both `ws://` and `wss://` when the implementation advertises the matching binding. The built-in `openshell/regex` advertises both bindings and applies its fixed patterns to UTF-8 text. A host-matched HTTP-only attachment can inspect the upgrade GET but does not join the WebSocket chain; look for `binding_not_selected` coverage. Binary messages pass under both `on_error` modes and active stages emit `unsupported_message_type` coverage; upstream-to-client messages remain uninspected. A broken fail-open WebSocket stage is disabled for the rest of that connection; inspect sandbox OCSF logs for `openshell.middleware.websocket_stage_disabled`. ### Step 5: Push the updated policy @@ -276,6 +419,16 @@ Only `network_policies` can be modified at runtime. If `filesystem_policy`, `lan openshell policy set dev --policy current-policy.yaml --wait ``` +The gateway validates the complete effective candidate—including attached +provider-profile policy—before it stores a direct update, incremental merge, +approved proposal, provider attachment, or profile update that affects attached +sandboxes. An ambiguity failure returns `FAILED_PRECONDITION`; the rejected +candidate does not create a policy revision or partially update affected +sandboxes. The same fail-closed response applies when `credential_signing` +does not have an attached AWS profile whose credential boundary covers the +endpoint, or an explicit binding to an endpointless AWS profile. Fix the +conflicting endpoint selectors or credential source and submit again. + The `--wait` flag blocks until the sandbox confirms the policy is loaded (polls every second). Exit codes: - **0**: Policy loaded successfully - **1**: Policy load failed @@ -307,27 +460,56 @@ Fetch a specific historical revision: openshell policy get dev --rev 3 --full ``` +Gateway-global policy commands use `--global` and require confirmation unless `--yes` is supplied: + +```bash +openshell policy get --global --full +openshell policy set --global --policy ./global-policy.yaml +openshell policy list --global +openshell policy delete --global +``` + +Avoid `--yes` during interactive work. A global policy locks policy control for all sandboxes on the gateway. + +### Review agent-authored rule proposals + +Sandboxes created with `--approval-mode manual` place every proposal in the review inbox. `auto` approves only proposals with an empty prover delta; findings still require review. + +```bash +openshell rule get dev --status pending +openshell rule approve dev --chunk-id +openshell rule reject dev --chunk-id --reason "too broad" +openshell rule history dev +``` + +Review the proposed scope and prover findings before approval. Treat `rule approve-all --include-security-flagged` as a high-risk bulk action. + --- ## Workflow 5: BYOC (Bring Your Own Container) Build a custom container image and run it as a sandbox. -### Step 1: Create a sandbox from a Dockerfile +### Create a sandbox from a Dockerfile ```bash openshell sandbox create --from ./Dockerfile --name my-app ``` -The `--from` flag accepts a Dockerfile path, a directory containing a Dockerfile, a full image reference (e.g. `myregistry.com/img:tag`), or a community sandbox name (e.g. `openclaw`). +The `--from` flag accepts a Dockerfile path, a directory containing a Dockerfile, a full image reference such as `myregistry.com/img:tag`, or a community sandbox name such as `ollama`. -When given a Dockerfile or directory, the image is built locally via Docker and delivered through the selected compute driver. Docker and Podman-backed gateways can use local images directly. Kubernetes gateways usually need the image available to the cluster through a registry or driver-supported image push path. +Local Dockerfile and directory builds require a local gateway because the CLI builds through the local Docker daemon. Use a registry image reference for remote gateways. Bare community names resolve under `ghcr.io/nvidia/openshell-community/sandboxes` unless `OPENSHELL_COMMUNITY_REGISTRY` overrides the prefix. -When `--from` is specified, the CLI: -- Clears default `run_as_user`/`run_as_group` (custom images may not have the `sandbox` user) -- Uses a supervisor bootstrap pattern (init container copies the sandbox supervisor into a shared volume) +For Docker and Podman gateways, custom images should declare a non-root OCI +`USER`. Each explicit `process.run_as_user` or `process.run_as_group` policy +field wins independently; omitted fields fall back to the image declaration. +An image with no `USER` fails before readiness unless policy supplies both +fields. Explicit numeric fields may use any UID/GID from `1` through +`4294967294`; `0` is root and `4294967295` is the invalid identity sentinel. +Warn users that low IDs can inherit permissions from matching accounts, image +files, mounted volumes, or devices. -### Step 2: Forward ports (if the container runs a service) +### Forward ports ```bash # Foreground (blocks) @@ -339,168 +521,183 @@ openshell forward start 8080 my-app -d The service is now reachable at `localhost:8080`. -### Step 3: Manage port forwards +Manage or iterate on the sandbox: ```bash -# List active forwards openshell forward list - -# Stop a forward openshell forward stop 8080 my-app -``` - -### Step 4: Iterate - -To update the container: - -```bash openshell sandbox delete my-app openshell sandbox create --from ./Dockerfile --name my-app --forward 8080 ``` -### Shortcut: Create with port forward in one command +Create and forward in one command: ```bash openshell sandbox create --from ./Dockerfile --forward 8080 -- ./start-server.sh ``` -The `--forward` flag starts a background port forward before the command runs, so the service is reachable immediately. - -### Limitations - -- Distroless / `FROM scratch` images are not supported (the supervisor needs glibc, `/proc`, and a shell) -- Missing `iproute2` or required capabilities blocks startup in proxy mode - ---- +The `--forward` flag starts a background port forward before the command runs. ## Workflow 6: Agent-Assisted Sandbox Session -This workflow supports a human working in a sandbox while an agent monitors activity and refines the policy in parallel. +Support a human working in a sandbox while an agent monitors activity and refines the policy in parallel. -### Step 1: Create sandbox with providers and keep alive +Create the sandbox and keep it alive: ```bash openshell sandbox create \ --name work-session \ --provider github \ --provider claude \ - --policy ./dev-policy.yaml \ - # sandbox create keeps the sandbox alive by default + --policy ./dev-policy.yaml ``` -### Step 2: User connects in a separate shell - -Tell the user to run: +Tell the user to connect in another shell: ```bash openshell sandbox connect work-session +openshell sandbox connect work-session --editor vscode ``` -Or for VS Code: - -```bash -openshell sandbox ssh-config work-session >> ~/.ssh/config -# Then connect via VS Code Remote-SSH to the host "work-session" -``` - -### Step 3: Agent monitors logs - -While the user works, monitor the sandbox logs: +Monitor denied activity: ```bash openshell logs work-session --tail --source sandbox --level warn ``` -Watch for `deny` actions that indicate the user's work is being blocked by policy. - -### Step 4: Agent refines policy - -When denied actions are observed: +When denied actions appear: 1. Prefer incremental updates for additive network changes: `openshell policy update work-session --add-endpoint api.github.com:443:read-only:rest:enforce --binary /usr/bin/gh --wait` `openshell policy update work-session --add-allow 'api.github.com:443:POST:/repos/*/issues' --wait` -2. Use full YAML replacement when the change is broad or touches non-network fields: + + A rule authorizes every binary it lists to reach every endpoint it lists, so + an update that adds a binary or an endpoint to an existing rule must declare + that rule's whole binary and endpoint scope. The gateway rejects an update + that would grant a binary-to-endpoint pair the update never asked for, and + the error names the binaries still missing. To grant one binary access to + only part of a rule's endpoints, send the narrow authorization under its own + `--rule-name`; it stays on its own rule instead of folding into the broader + one. + + `--add-allow` and `--add-deny` select an endpoint by host and port alone. If + that host and port appears in more than one rule, or twice in one rule under + different paths, the update is rejected as ambiguous. Fall back to full YAML + replacement for those endpoints. +2. Use full YAML replacement for broad changes or non-network fields, including + any change that would otherwise require restating a large existing scope: `openshell policy get work-session --full > policy.yaml` - Modify the policy to allow the blocked actions (use `generate-sandbox-policy` skill for content) + Modify the policy with the `generate-sandbox-policy` skill. `openshell policy set work-session --policy policy.yaml --wait` -3. Verify: `openshell policy list work-session` - -The user does not need to disconnect -- policy updates are hot-reloaded within ~30 seconds (or immediately when using `--wait`, which polls for confirmation). +3. Verify with `openshell policy list work-session`. -### Step 5: Clean up when done +The user does not need to disconnect. Policy updates are hot-reloaded; `--wait` blocks until the sandbox confirms the revision or the timeout expires. Delete the sandbox when the session ends: ```bash openshell sandbox delete work-session ``` ---- - -## Workflow 7: Gateway Inference - -Configure the gateway's managed inference route for `inference.local`. +## Workflow 7: Managed Inference -### Set gateway inference +Configure the user-facing `inference.local` route or the system inference route used by platform functions. -First ensure the provider record exists: +Ensure the provider exists, then set the route: ```bash openshell provider list -``` - -Then point gateway inference at that provider and model: - -```bash openshell inference set \ --provider nvidia \ --model nvidia/nemotron-3-nano-30b-a3b ``` -This updates the gateway-managed `inference.local` route. There is no per-route create/list/update/delete workflow for sandbox inference. +This updates the managed `inference.local` route. Endpoint verification runs before the route is saved. Use `--no-verify` only when verification is intentionally impossible, and use `--timeout SECONDS` to configure the request timeout. Add `--system` to `set` or `update` for the platform-only system route. -### Inspect current inference config +Inspect both configurations: ```bash openshell inference get +openshell inference get --system ``` -### How sandboxes use it - -- Agents send HTTPS requests to `inference.local`. -- The sandbox intercepts those requests locally and routes them through the gateway inference config. -- Sandbox policy is separate from gateway inference configuration. - ---- +Agents send HTTPS requests to `inference.local`; the sandbox intercepts them and routes them through the configured inference route. Sandbox policy remains separate from inference route configuration. ## Workflow 8: Gateway Management -### List and switch gateways +List, switch, and verify gateways: ```bash -openshell gateway select # See all gateways (no args shows list) -openshell gateway select production # Switch active gateway -openshell status # Verify connectivity +openshell gateway select +openshell gateway list --output json +openshell gateway select production +openshell gateway info --name production +openshell status ``` -### Registration +Register or remove gateways: ```bash openshell gateway add http://127.0.0.1:8080 --local --name local openshell gateway add https://gateway.example.com --name production -openshell gateway remove local # Remove local registration +openshell gateway remove local ``` -### Platform-specific deployment inspection +`https://` registrations default to edge authentication. Use `gateway login` and `gateway logout` to refresh or clear stored authentication. For an OIDC gateway, supply `--oidc-issuer` and, when needed, `--oidc-client-id`, `--oidc-audience`, and `--oidc-scopes`. For remote mTLS gateways, use `--remote USER@HOST` or an `ssh://` endpoint. + +For one-off automation, `--gateway-endpoint URL` connects directly without stored metadata. Limit `--gateway-insecure` to explicitly trusted development endpoints. + +Inspect a Kubernetes deployment: ```bash -# Inspect a Kubernetes Helm release and gateway pod helm -n openshell status openshell -kubectl -n openshell get pods,svc -kubectl -n openshell logs statefulset/openshell --tail=100 +kubectl -n openshell get deployment,statefulset,pods,svc +kubectl -n openshell logs deployment/openshell -c openshell-gateway --tail=100 +kubectl -n openshell logs statefulset/openshell -c openshell-gateway --tail=100 ``` For Docker, Podman, and VM-backed gateways, inspect the gateway process or container logs and the selected runtime directly. +## Workflow 9: Settings Management + +Manage sandbox-scoped or gateway-global settings: + +```bash +openshell settings get work-session +openshell settings set work-session --key ocsf_json_enabled --value true +openshell settings delete work-session --key ocsf_json_enabled + +openshell settings get --global --json +openshell settings set --global --key providers_v2_enabled --value true +``` + +Global mutations prompt for confirmation. Use `--yes` only in reviewed automation. + +`policy_validation_failure_mode` is gateway startup configuration, not a +mutable `openshell settings` key. Set it under `[openshell.gateway]` in +`gateway.toml` and restart the gateway. The security-first default is +`fail_closed`; `retain_last_valid` is an explicit availability tradeoff. OCSF +configuration events state whether the previous generation is active after a +runtime validation failure. + +## Workflow 10: Service Access + +Use `forward` for local access and `service` for a gateway-managed HTTP endpoint: + +```bash +# SSH-based same-port forwarding; optional bind address is accepted. +openshell forward start 127.0.0.1:8080 my-app -d + +# gRPC relay to a loopback TCP service, with an optional dynamic local port. +openshell forward service my-app --target-port 8000 --local 127.0.0.1:0 + +# Expose and manage an HTTP service through the gateway. +openshell service expose my-app 8080 web +openshell service list my-app +openshell service get my-app web +openshell service delete my-app web +``` + +Prefer loopback binds unless the user explicitly needs LAN-visible local access. + --- ## Self-Teaching via `--help` @@ -517,7 +714,7 @@ The CLI help is always authoritative. If the help output contradicts this skill, ```bash $ openshell sandbox --help -# Shows: create, get, list, delete, connect, upload, download, ssh-config, image +# Shows: create, get, list, stop, start, delete, exec, connect, upload, download, ssh-config, provider $ openshell sandbox upload --help # Shows: positional arguments (name, path, dest), usage examples @@ -530,25 +727,37 @@ $ openshell sandbox upload --help | Task | Command | |------|---------| | Register local port-forwarded gateway | `openshell gateway add http://127.0.0.1:8080 --local --name local` | -| Check gateway health | `openshell status` | +| Check gateway health and authentication | `openshell status` | +| Show authenticated identity and subject | `openshell whoami` | | List/switch gateways | `openshell gateway select [name]` | +| Connect directly to a gateway | `openshell --gateway-endpoint status` | | Create sandbox (interactive) | `openshell sandbox create` | | Create sandbox with tool | `openshell sandbox create -- claude` | +| Create sandbox with GPUs | `openshell sandbox create --gpu 1` | | Create with custom policy | `openshell sandbox create --policy ./p.yaml` | | Connect to sandbox | `openshell sandbox connect ` | +| Stop sandbox compute | `openshell sandbox stop [name]` | +| Start sandbox compute | `openshell sandbox start [name]` | +| Execute in sandbox | `openshell sandbox exec --name -- ` | | Stream live logs | `openshell logs --tail` | | Incremental policy update | `openshell policy update --add-endpoint host:443:read-only:rest:enforce --binary /usr/bin/curl --wait` | | Pull current policy | `openshell policy get --full > p.yaml` | | Push updated policy | `openshell policy set --policy p.yaml --wait` | | Policy revision history | `openshell policy list ` | +| View global policy | `openshell policy get --global --full` | +| Review proposed rules | `openshell rule get --status pending` | | Create sandbox from Dockerfile | `openshell sandbox create --from ./Dockerfile` | | Forward a port | `openshell forward start -d` | +| Expose an HTTP service | `openshell service expose [service]` | | Upload files to sandbox | `openshell sandbox upload ` | | Download files from sandbox | `openshell sandbox download ` | | Create provider | `openshell provider create --name N --type T --from-existing` | | List providers | `openshell provider list` | -| Configure gateway inference | `openshell inference set --provider P --model M` | -| View gateway inference | `openshell inference get` | +| Discover provider profiles | `openshell provider list-profiles` | +| List attached providers | `openshell sandbox provider list ` | +| View settings | `openshell settings get [name]` | +| Configure managed inference | `openshell inference set --provider P --model M` | +| View managed inference | `openshell inference get` | | Delete sandbox | `openshell sandbox delete ` | | Remove gateway registration | `openshell gateway remove ` | | Self-teach any command | `openshell --help` | @@ -557,7 +766,7 @@ $ openshell sandbox upload --help | Skill | When to use | |-------|------------| -| `generate-sandbox-policy` | Creating or modifying policy YAML content (network rules, L7 inspection, access presets, endpoint configuration) | +| `generate-sandbox-policy` | Creating or modifying policy YAML content (network rules, L7 inspection, access presets, endpoint configuration, and network middleware) | | `debug-openshell-cluster` | Diagnosing gateway deployment, runtime, or health failures | | `debug-inference` | Diagnosing `inference.local`, host-backed local inference, and provider base URL issues | | `tui-development` | Developing features for the OpenShell TUI (`openshell term`) | diff --git a/.agents/skills/openshell-cli/cli-reference.md b/.agents/skills/openshell-cli/cli-reference.md index 7998502326..2cd5881ab9 100644 --- a/.agents/skills/openshell-cli/cli-reference.md +++ b/.agents/skills/openshell-cli/cli-reference.md @@ -10,13 +10,19 @@ Quick-reference for the `openshell` command-line interface. For workflow guidanc |------|-------------| | `-v`, `--verbose` | Increase verbosity (`-v` = info, `-vv` = debug, `-vvv` = trace) | | `-g`, `--gateway ` | Gateway to operate on. Also settable via `OPENSHELL_GATEWAY` env var. Falls back to active gateway in `~/.config/openshell/active_gateway`. | +| `--gateway-endpoint ` | Connect directly to a gateway endpoint without looking up stored metadata. Also settable via `OPENSHELL_GATEWAY_ENDPOINT`. | +| `--gateway-insecure` | Skip TLS certificate verification. Also settable via `OPENSHELL_GATEWAY_INSECURE`; use only for trusted development endpoints. | ## Environment Variables | Variable | Description | |----------|-------------| | `OPENSHELL_GATEWAY` | Override active gateway name (same as `--gateway`) | +| `OPENSHELL_GATEWAY_ENDPOINT` | Connect directly to a gateway endpoint (same as `--gateway-endpoint`) | +| `OPENSHELL_GATEWAY_INSECURE` | Skip TLS verification when set (same as `--gateway-insecure`) | | `OPENSHELL_SANDBOX_POLICY` | Path to default sandbox policy YAML (fallback when `--policy` is not provided) | +| `OPENSHELL_COMMUNITY_REGISTRY` | Override the community sandbox image registry prefix used by `sandbox create --from ` | +| `OPENSHELL_THEME` | TUI theme: `auto`, `dark`, or `light` | --- @@ -33,35 +39,73 @@ openshell │ ├── list │ └── select [name] ├── status +├── whoami [--output ] ├── inference │ ├── set --provider --model │ ├── update [--provider] [--model] │ └── get ├── sandbox │ ├── create [opts] [-- CMD...] -│ ├── get +│ ├── get [name] │ ├── list [opts] -│ ├── delete ... -│ ├── connect +│ ├── stop [name] +│ ├── start [name] +│ ├── delete [name]... [--all] +│ ├── exec [--name ] [opts] -- CMD... +│ ├── connect [name] [--editor ] │ ├── upload [dest] │ ├── download [dest] -│ ├── ssh-config -│ └── image -│ └── push [opts] +│ ├── ssh-config [name] +│ └── provider +│ ├── list [name] +│ ├── attach +│ └── detach ├── forward -│ ├── start [-d] -│ ├── stop -│ └── list -├── logs [opts] +│ ├── start [name] [-d] +│ ├── stop [name] +│ ├── list +│ └── service [name] --target-port [opts] +├── service +│ ├── expose [service] +│ ├── list [sandbox] +│ ├── get [service] +│ └── delete [service] +├── logs [name] [opts] ├── policy -│ ├── set --policy [--wait] -│ ├── get [--full] -│ └── list +│ ├── set [name] --policy [--global] [--wait] +│ ├── update [name] [opts] +│ ├── get [name] [--full|--base] [--global] +│ ├── list [name] [--global] +│ ├── delete --global +│ └── prove --policy --credentials [opts] +├── settings +│ ├── get [name] [--global] +│ ├── set [name] --key --value [--global] +│ └── delete [name] --key [--global] +├── rule (advanced; hidden from top-level help) +│ ├── get [name] [--status ] +│ ├── approve [name] --chunk-id +│ ├── reject [name] --chunk-id [--reason ] +│ ├── approve-all [name] [--include-security-flagged] +│ ├── clear [name] +│ └── history [name] ├── provider │ ├── create --name --type [opts] +│ ├── refresh +│ │ ├── status [opts] +│ │ ├── configure [opts] +│ │ ├── rotate --credential-key +│ │ └── delete --credential-key │ ├── get │ ├── list [opts] -│ ├── update --type [opts] +│ ├── list-profiles [opts] +│ ├── profile +│ │ ├── export [opts] +│ │ ├── import (--file |--from ) +│ │ ├── update --file +│ │ ├── lint (--file |--from ) +│ │ └── delete +│ ├── update [opts] │ └── delete ... ├── doctor │ └── check @@ -81,13 +125,20 @@ Register an existing gateway endpoint. | Flag | Description | |------|-------------| | `--name ` | Gateway name | -| `--local` | Register a local endpoint, commonly a trusted port-forward | -| `--remote ` | Register a remote gateway associated with an SSH destination | +| `--local` | Register a local mTLS gateway; with HTTP, store a local plaintext registration | +| `--remote ` | Register a remote mTLS gateway over SSH; with HTTP, store a remote plaintext registration | +| `--oidc-issuer ` | Register an OIDC-authenticated gateway | +| `--oidc-client-id ` | OIDC client ID (default: `openshell-cli`; requires `--oidc-issuer`) | +| `--oidc-audience ` | OIDC API audience (requires `--oidc-issuer`) | +| `--oidc-scopes ` | Space-separated OAuth2 scopes (requires `--oidc-issuer`) | Examples: - `openshell gateway add http://127.0.0.1:8080 --local --name local` - `openshell gateway add https://gateway.example.com --name production` +- `openshell gateway add ssh://user@gateway.example.com:8080 --name remote` + +An `http://` endpoint is direct plaintext. A plain `https://` endpoint uses edge authentication. `--local` and `--remote` select mTLS registration modes when used with HTTPS; required certificates must already exist. An `ssh://` endpoint is shorthand for a remote gateway. ### `openshell gateway remove [name]` @@ -95,7 +146,11 @@ Remove a local gateway registration. This removes CLI metadata and stored auth t ### `openshell gateway login [name]` -Refresh browser-based authentication for a gateway behind an edge proxy. +Refresh browser-based authentication for an edge-authenticated or OIDC gateway. + +### `openshell gateway logout [name]` + +Clear locally stored OIDC or edge credentials for a gateway. ### `openshell gateway info` @@ -107,7 +162,11 @@ Show gateway details: endpoint, auth mode, and remote host metadata when present ### `openshell gateway select [name]` -Set the active gateway. Writes to `~/.config/openshell/active_gateway`. When called without arguments, lists all registered gateways with the active one marked with `*`. +Set the active gateway. Writes to `~/.config/openshell/active_gateway`. Without a name, opens an interactive chooser on a TTY or lists gateways in non-interactive mode. + +### `openshell gateway list` + +List registered gateways and mark the active one. `--output table|yaml|json` selects the format. --- @@ -125,7 +184,19 @@ package-managed or Helm gateways, use `systemctl`, `journalctl`, `kubectl`, and ### `openshell status` -Show server connectivity and version for the active gateway. +Show server connectivity, authentication status, and version for the active +gateway. Connectivity uses the public health RPC; authentication is checked +with the protected gateway-info capability query and can fail while the gateway +remains connected. + +### `openshell whoami` + +Show the authenticated user identity: subject, display name, roles, scopes, and +identity provider. Requires an authenticated gateway connection. + +| Flag | Description | +|------|-------------| +| `--output ` | Output format: `table` (default), `json`, or `yaml` | --- @@ -133,103 +204,149 @@ Show server connectivity and version for the active gateway. ### `openshell sandbox create [OPTIONS] [-- COMMAND...]` -Create a sandbox through the active gateway, wait for readiness, then connect or execute the trailing command. +Create a sandbox through the selected gateway, wait for readiness, then connect, open an editor, or execute the trailing command. | Flag | Description | |------|-------------| | `--name ` | Sandbox name (auto-generated if omitted) | -| `--from ` | Sandbox source: community name, Dockerfile path, directory, or image reference (BYOC) | -| `--upload [:]` | Upload local files into sandbox (default dest: `/sandbox`) | -| `--no-keep` | Delete sandbox after the initial command or shell exits | +| `--from ` | Community name, Dockerfile path, directory, or image reference (BYOC) | +| `--no-keep` | Delete the sandbox after the initial command or shell exits | +| `--editor vscode|cursor` | Launch a remote editor and keep the sandbox alive | +| `--gpu [COUNT]` | Request the driver's default GPU selection or a specific count | +| `--cpu ` | CPU limit (for example: `500m`, `1`, `2.5`) | +| `--memory ` | Memory limit (for example: `512Mi`, `4Gi`, `8G`) | +| `--driver-config-json ` | Experimental driver-keyed configuration object | | `--provider ` | Provider to attach (repeatable) | -| `--policy ` | Path to custom policy YAML | -| `--cpu ` | CPU amount for the sandbox (for example: `500m`, `1`, `2.5`) | -| `--memory ` | Memory amount for the sandbox (for example: `512Mi`, `4Gi`, `8G`) | -| `--forward ` | Forward local port to sandbox (keeps the sandbox alive) | -| `--tty` | Force pseudo-terminal allocation | -| `--no-tty` | Disable pseudo-terminal allocation | -| `--auto-providers` | Auto-create missing providers from local credentials (skips interactive prompt) | -| `--no-auto-providers` | Never auto-create providers; skip missing providers silently | -| `[-- COMMAND...]` | Command to execute (defaults to interactive shell) | - -### `openshell sandbox get ` - -Show sandbox details (id, name, namespace, phase) and the **active** policy from the gateway (same source whether policy is sandbox-scoped or global). Metadata includes **Policy source** (`sandbox` or `global`) and **Revision** (global policy row when source is global, otherwise sandbox policy row). +| `--policy ` | Custom policy YAML; overrides the built-in default and `OPENSHELL_SANDBOX_POLICY` | +| `--forward <[BIND:]PORT>` | Start a local port forward and keep the sandbox alive | +| `--tty`, `--no-tty` | Force or disable pseudo-terminal allocation | +| `--auto-providers` | Auto-create missing providers from local credentials | +| `--no-auto-providers` | Never auto-create providers; error if a required provider is missing | +| `--label ` | Attach a label (repeatable) | +| `--env ` | Inject an environment variable (repeatable) | +| `--approval-mode manual|auto` | Handle agent-authored policy proposals; default: `manual` | +| `--upload [:]` | Upload local files to the working directory or an explicit destination (repeatable) | +| `--no-git-ignore` | Disable `.gitignore` filtering for `--upload` | +| `[-- COMMAND...]` | Initial command (defaults to an interactive shell) | + +### `openshell sandbox get [name]` + +Show sandbox details and the active policy. Metadata identifies sandbox or global policy source and the corresponding revision. The name defaults to the last-used sandbox. | Flag | Description | |------|-------------| -| `--policy-only` | Print only the active policy YAML to stdout (same policy as above; use for scripts and piping) | +| `--policy-only` | Print only the active policy YAML to stdout | ### `openshell sandbox list` -List sandboxes in a table. - | Flag | Default | Description | |------|---------|-------------| -| `--limit ` | 100 | Max sandboxes to return | +| `--limit ` | 100 | Maximum sandboxes | | `--offset ` | 0 | Pagination offset | | `--ids` | false | Print only sandbox IDs | | `--names` | false | Print only sandbox names | +| `--selector ` | none | Filter by `key1=value1,key2=value2` | +| `--output table|yaml|json` | `table` | Output format | -### `openshell sandbox delete ...` +### `openshell sandbox delete [NAME]...` -Delete one or more sandboxes by name. Stops any background port forwards. +Delete one or more named sandboxes, or use `--all`. Deletion stops background port forwards. -### `openshell sandbox connect ` +### `openshell sandbox stop [name]` -Open an interactive SSH shell to a sandbox. +Stop sandbox compute while retaining the sandbox and persistent workspace. The +name defaults to the last-used sandbox. The command stops background forwards +and waits for the `Stopped` phase. -### `openshell sandbox upload [dest]` +### `openshell sandbox start [name]` + +Start a stopped sandbox and wait for `Ready`. The name defaults to the +last-used sandbox. -Upload local files to a sandbox using tar-over-SSH. +### `openshell sandbox exec [OPTIONS] -- COMMAND...` -| Argument | Default | Description | -|----------|---------|-------------| -| `` | -- | Sandbox name (required) | -| `` | -- | Local path to upload (required) | -| `[dest]` | `/sandbox` | Destination path in sandbox | +Execute a command through the gRPC exec endpoint, stream its output, and exit with the remote command's exit code. + +| Flag | Default | Description | +|------|---------|-------------| +| `-n`, `--name ` | last-used | Sandbox name | +| `--workdir ` | none | Working directory in the sandbox | +| `--timeout ` | 0 | Command timeout; `0` disables it | +| `--tty`, `--no-tty` | auto | Force or disable a pseudo-terminal | +| `--env ` | none | Command environment variable (repeatable) | + +### `openshell sandbox connect [name]` + +Open an interactive SSH shell. The name defaults to the last-used sandbox. `--editor vscode|cursor` launches a supported remote editor instead. + +### `openshell sandbox upload [dest]` + +Upload files using tar-over-SSH. The CLI discovers the canonical remote working directory when the destination is omitted. A named directory merges into an existing directory of the same name, overwriting matching entries without deleting unrelated entries. `.gitignore` filtering is enabled unless `--no-git-ignore` is passed. ### `openshell sandbox download [dest]` -Download files from a sandbox using tar-over-SSH. +Download files using tar-over-SSH. The sandbox source may be relative to the canonical remote working directory or an absolute path within it. The local destination defaults to `.`. + +### `openshell sandbox ssh-config [name]` -| Argument | Default | Description | -|----------|---------|-------------| -| `` | -- | Sandbox name (required) | -| `` | -- | Sandbox path to download (required) | -| `[dest]` | `.` | Local destination path | +Print an SSH config `Host` block. The name defaults to the last-used sandbox. -### `openshell sandbox ssh-config ` +### `openshell sandbox provider` -Print an SSH config `Host` block for a sandbox. Useful for VS Code Remote-SSH. +Manage providers on an existing sandbox: + +- `openshell sandbox provider list [name]` +- `openshell sandbox provider attach ` +- `openshell sandbox provider detach ` --- ## Port Forwarding Commands -### `openshell forward start ` +### `openshell forward start [name]` Start forwarding a local port to a sandbox. | Flag | Description | |------|-------------| -| `` | Port number (used as both local and remote) | -| `` | Sandbox name | +| `` | `[bind_address:]port`; the port is used locally and remotely | +| `[name]` | Sandbox name (defaults to last-used) | | `-d`, `--background` | Run in background | -### `openshell forward stop ` +### `openshell forward stop [name]` -Stop a background port forward. +Stop a background port forward. When the sandbox name is omitted, it is inferred from active forwards. ### `openshell forward list` List all active port forwards (sandbox, port, PID, status). +### `openshell forward service [name] --target-port ` + +Forward a local TCP port to a loopback service inside a sandbox over the gRPC relay. + +| Flag | Default | Description | +|------|---------|-------------| +| `--target-port ` | required | Service port inside the sandbox | +| `--target-host ` | `127.0.0.1` | Loopback service host | +| `--local <[BIND:]PORT>` | target port | Local bind; port `0` requests dynamic assignment | + +--- + +## Service Commands + +Gateway-managed HTTP service endpoints: + +- `openshell service expose [service]` +- `openshell service list [sandbox] [--limit N] [--offset N]` +- `openshell service get [service]` +- `openshell service delete [service]` + --- ## Logs Command -### `openshell logs ` +### `openshell logs [name]` View sandbox logs. Supports one-shot and streaming. @@ -241,20 +358,22 @@ View sandbox logs. Supports one-shot and streaming. | `--source ` | `all` | Filter: `gateway`, `sandbox`, or `all` (repeatable) | | `--level ` | none | Minimum level: `error`, `warn`, `info`, `debug`, `trace` | +The sandbox name defaults to the last-used sandbox. + --- ## Policy Commands -### `openshell policy update ` +### `openshell policy update [name]` Incrementally merge live network policy changes into the current sandbox policy. Multiple flags in one invocation are applied as one atomic batch and create at most one new revision. | Flag | Default | Description | |------|---------|-------------| -| `--add-endpoint ` | repeatable | `host:port[:access[:protocol[:enforcement]]]`. Adds or merges an endpoint. `access`: `read-only`, `read-write`, `full`. `protocol`: `rest`, `sql`. `enforcement`: `enforce`, `audit`. | +| `--add-endpoint ` | repeatable | `host:port[:access[:protocol[:enforcement[:options]]]]`. Adds or merges an endpoint. | | `--remove-endpoint ` | repeatable | `host:port`. Removes the endpoint or just the requested port from a multi-port endpoint. | -| `--add-allow ` | repeatable | `host:port:METHOD:path_glob`. Adds REST allow rules to an existing `protocol: rest` endpoint. | -| `--add-deny ` | repeatable | `host:port:METHOD:path_glob`. Adds REST deny rules to an existing `protocol: rest` endpoint that already has an allow base. | +| `--add-allow ` | repeatable | `host:port:METHOD:path_glob`. Adds REST or WebSocket allow rules. | +| `--add-deny ` | repeatable | `host:port:METHOD:path_glob`. Adds REST or WebSocket deny rules. | | `--remove-rule ` | repeatable | Deletes a named network rule. | | `--binary ` | repeatable | Adds binaries to each `--add-endpoint` rule. Valid only with `--add-endpoint`. | | `--rule-name ` | none | Overrides the generated rule name. Valid only when exactly one `--add-endpoint` is provided. | @@ -264,44 +383,93 @@ Incrementally merge live network policy changes into the current sandbox policy. Notes: -- `--add-allow` and `--add-deny` currently operate only on `protocol: rest` endpoints. +- The sandbox name defaults to the last-used sandbox. +- `--add-endpoint` options are comma-separated: `allowed-ip=`, `websocket-credential-rewrite`, `request-body-credential-rewrite`, and `allow-uninspected-credentials`. The last option is a security-sensitive exception for provider-credentialed L4-only, `tls: skip`, or otherwise uninspectable traffic. +- `--add-allow` and `--add-deny` operate on REST and WebSocket endpoints. Use full YAML for JSON-RPC, MCP, SQL, or other policy structure. - `--wait` cannot be combined with `--dry-run`. - Use `policy set` when replacing the full policy or changing static sections. -### `openshell policy set --policy ` +### `openshell policy set [name] --policy ` Replace the full policy on a live sandbox. Only the dynamic `network_policies` field can be changed at runtime. | Flag | Default | Description | |------|---------|-------------| | `--policy ` | -- | Path to policy YAML (required) | +| `--global` | false | Apply as the gateway-global policy | +| `--yes` | false | Skip confirmation for a global update | | `--wait` | false | Wait for sandbox to confirm policy is loaded | | `--timeout ` | 60 | Timeout for `--wait` | Exit codes with `--wait`: 0 = loaded, 1 = failed, 124 = timeout. -### `openshell policy get ` +### `openshell policy get [name]` -Show current active policy for a sandbox. +Show the current effective sandbox policy or stored global policy. | Flag | Default | Description | |------|---------|-------------| -| `--rev ` | 0 (latest) | Show a specific revision | -| `--full` | false | Print the full policy as YAML (round-trips with `--policy` input) | +| `--rev ` | 0 | Show a stored revision; `0` shows the current effective policy | +| `--full` | false | Include the effective policy payload and provider-composed entries | +| `--base` | false | Include the base policy payload without provider-composed entries | +| `--output table|json` | `table` | Output format | +| `--global` | false | Show the global policy revision | -### `openshell policy list ` +### `openshell policy list [name]` List policy revision history (version, hash, status, created, error). | Flag | Default | Description | |------|---------|-------------| | `--limit ` | 20 | Max revisions to return | +| `--global` | false | List global policy revisions | + +### `openshell policy delete --global` + +Delete the global policy lock and restore sandbox-level policy control. `--yes` skips confirmation. + +### `openshell policy prove` + +Prove policy properties or find counterexamples. + +| Flag | Description | +|------|-------------| +| `--policy ` | Policy YAML (required) | +| `--credentials ` | Credential descriptor YAML (required) | +| `--registry ` | Capability registry directory (defaults to bundled) | +| `--accepted-risks ` | Accepted-risks YAML | +| `--compact` | One-line-per-finding output | + +### `openshell rule` (advanced) + +Review agent-authored network rule proposals. This command group is intentionally hidden from top-level help but is part of the policy-advisor workflow. + +- `openshell rule get [name] [--status pending|approved|rejected]` +- `openshell rule approve [name] --chunk-id ` +- `openshell rule reject [name] --chunk-id [--reason ]` +- `openshell rule approve-all [name] [--include-security-flagged]` +- `openshell rule clear [name]` +- `openshell rule history [name]` + +Sandbox names default to the last-used sandbox. Bulk approval of security-flagged proposals requires explicit `--include-security-flagged`. + +--- + +## Settings Commands + +Settings support sandbox and gateway-global scopes: + +- `openshell settings get [name] [--global] [--json]` +- `openshell settings set [name] --key --value [--global] [--yes]` +- `openshell settings delete [name] --key [--global] [--yes]` + +Sandbox names default to the last-used sandbox. Global mutations prompt unless `--yes` is passed. --- ## Provider Commands -Supported provider types: `claude`, `opencode`, `codex`, `generic`, `nvidia`, `gitlab`, `github`, `outlook`. +Provider types are defined by built-in and custom provider profiles. Use `openshell provider list-profiles` to discover the selected gateway's current inventory. ### `openshell provider create --name --type ` @@ -311,10 +479,14 @@ Create a provider configuration. |------|-------------| | `--name ` | Provider name (required) | | `--type ` | Provider type (required) | -| `--from-existing` | Load credentials from local state (mutually exclusive with `--credential`) | +| `--from-existing` | Load credentials and config from local state | | `--credential KEY[=VALUE]` | Credential pair. Bare `KEY` reads from env var. Repeatable. | +| `--from-gcloud-adc` | Load a compatible credential from gcloud Application Default Credentials | +| `--runtime-credentials` | Resolve required credentials at runtime in the gateway or sandbox | | `--config KEY=VALUE` | Config key/value pair. Repeatable. | +Exactly one credential source is required. Credential-source flags conflict with one another. + ### `openshell provider get ` Show provider details (id, name, type, credential keys, config keys). @@ -328,40 +500,80 @@ List providers in a table. | `--limit ` | 100 | Max providers | | `--offset ` | 0 | Pagination offset | | `--names` | false | Print only names | +| `--output table|yaml|json` | `table` | Output format | + +### `openshell provider update ` -### `openshell provider update --type ` +Update an existing provider without changing its type. -Update an existing provider. Same flags as `create`. +| Flag | Description | +|------|-------------| +| `--from-existing` | Rediscover local credentials and config | +| `--credential KEY[=VALUE]` | Update a credential (repeatable) | +| `--config KEY=VALUE` | Update config (repeatable) | +| `--credential-expires-at KEY=TIMESTAMP` | Set or clear credential expiry; accepts epoch milliseconds or RFC3339, and `0` clears | ### `openshell provider delete ...` Delete one or more providers by name. +### Provider profiles + +- `openshell provider list-profiles [--output table|yaml|json]` +- `openshell provider profile export [--output table|yaml|json]` +- `openshell provider profile import (--file |--from )` +- `openshell provider profile update --file ` +- `openshell provider profile lint (--file |--from )` +- `openshell provider profile delete ` + +### Provider credential refresh + +- `openshell provider refresh status [--credential-key ]` +- `openshell provider refresh rotate --credential-key ` +- `openshell provider refresh delete --credential-key ` + +`provider refresh configure ` accepts: + +| Flag | Description | +|------|-------------| +| `--credential-key ` | Injectable credential key (required) | +| `--strategy ` | `oauth2-refresh-token`, `oauth2-client-credentials`, or `google-service-account-jwt` | +| `--material KEY=VALUE` | Non-secret refresh material (repeatable) | +| `--secret-material-env KEY[=ENVVAR]` | Secret refresh material read from the CLI environment (repeatable) | +| `--secret-material-key KEY` | Mark a supplied material key secret (repeatable) | +| `--credential-expires-at TIMESTAMP` | Current credential expiry in epoch milliseconds or RFC3339 | + --- ## Inference Commands ### `openshell inference set` -Configure the managed gateway inference route used by `inference.local`. Both flags are required. +Configure the gateway's user-facing `inference.local` route or the platform-only system route. Provider and model are required. | Flag | Default | Description | |------|---------|-------------| | `--provider ` | -- | Provider record name (required) | | `--model ` | -- | Model identifier to use for generation requests (required) | +| `--system` | false | Configure the system inference route | +| `--no-verify` | false | Skip endpoint verification before saving | +| `--timeout ` | 0 | Request timeout; `0` uses the 60-second default | ### `openshell inference update` -Partially update the gateway inference configuration. Fetches the current config and applies only the provided overrides. At least one flag is required. +Partially update the selected inference route. | Flag | Default | Description | |------|---------|-------------| | `--provider ` | unchanged | Provider record name | | `--model ` | unchanged | Model identifier | +| `--system` | false | Target the system inference route | +| `--no-verify` | false | Skip endpoint verification before saving | +| `--timeout ` | unchanged | Request timeout; `0` uses the 60-second default | ### `openshell inference get` -Show the current gateway inference configuration. +Show both inference routes. `--system` shows only the system route. --- @@ -369,7 +581,7 @@ Show the current gateway inference configuration. ### `openshell term` -Launch the OpenShell interactive TUI. +Launch the OpenShell interactive TUI. `--theme auto|dark|light` overrides `OPENSHELL_THEME`. ### `openshell completions ` diff --git a/.agents/skills/review-github-pr/SKILL.md b/.agents/skills/review-github-pr/SKILL.md index 4901d19754..1058bfb3e9 100644 --- a/.agents/skills/review-github-pr/SKILL.md +++ b/.agents/skills/review-github-pr/SKILL.md @@ -111,6 +111,7 @@ Read through the full diff (and the PR description if available). Produce a summ - **Key Design Decisions**: Focus on _why_ something was done a particular way, not _what_ changed. Include `file_path:line_number` references. Examples: choice of algorithm, new abstraction introduced, API contract change, migration strategy. - **Notable Code**: Include only the most instructive or surprising snippets. Keep each snippet under 15 lines. Always include the file path above the code block. - **Potential Concerns**: Only include if there are genuine risks — missing error handling, breaking changes, performance implications, security issues. Do not fabricate concerns. +- **Agent infrastructure**: When the PR changes behavior, commands, or development workflows, use the `sync-agent-infra` maintenance map to check that related skills were updated. When it adds, removes, or renames skills or crates; changes workflow relationships or skill coverage; modifies issue or PR templates; or changes agent cross-references, apply the full consistency checklist. Report missing companion updates or drift under **Potential Concerns**. ## Step 5: Output diff --git a/.agents/skills/review-security-issue/SKILL.md b/.agents/skills/review-security-issue/SKILL.md index caac9fd1c1..b84e8b597a 100644 --- a/.agents/skills/review-security-issue/SKILL.md +++ b/.agents/skills/review-security-issue/SKILL.md @@ -11,6 +11,7 @@ Review an issue that outlines a security, vulnerability, or privacy concern. - The `gh` CLI must be authenticated (`gh auth status`) - You must be in a git repository with a GitHub remote +- The issue must have `topic:security`. In unattended queue mode it must also have `agent:plan-requested`; a direct user request to review a specific issue does not require that label. ## Agent Comment Marker @@ -40,7 +41,10 @@ gh issue view --json title,body,state,labels,author First, check the issue's labels from the metadata fetched in Step 1. -- **If the issue has the `state:agent-ready` label**, the issue has already been reviewed and is ready for implementation. There is no review to perform. Report to the user that this issue is already reviewed and marked as `state:agent-ready`, and suggest using the `fix-security-issue` skill instead. Stop. +- **If the issue has `agent:implementation-requested`**, the issue has already been reviewed and a human authorized remediation. There is no review to perform. Suggest using `fix-security-issue` and stop. +- **If `topic:security` is missing**, report that this specialized skill only reviews security issues and stop. +- **If this is queue mode and `agent:plan-requested` is missing**, report that the issue is not ready for unattended pickup and stop. +- **If the user directly requested review of this issue**, proceed even when `agent:plan-requested` is absent. Never add or offer to add the human-only request label. Next, fetch existing comments on the issue: @@ -133,15 +137,15 @@ EOF )" ``` -## Step 5: Add `state:review-ready` Label +## Step 5: Mark the Security Plan Ready -After posting the review comment (whether legitimate or not actionable), add the `state:review-ready` label to the issue: +After posting a legitimate-concern review with a remediation plan, replace `agent:plan-requested` with `agent:plan-ready` only when the request label was present: ```bash -gh issue edit --add-label "state:review-ready" +gh issue edit --remove-label "agent:plan-requested" --add-label "agent:plan-ready" ``` -This signals to humans and downstream skills (e.g., `fix-security-issue`) that the review is complete. +This signals that an unattended agent produced a remediation plan that awaits human review. For an unlabeled direct invocation, leave the `agent:*` labels unchanged. A later direct request can authorize remediation without `agent:implementation-requested`; unattended remediation still requires that label. For a not-actionable determination, remove `agent:plan-requested` if present, do not add another `agent:*` label, and report that a human should close the issue or record the risk decision. ## Step 6: Address Follow-up Comments @@ -163,7 +167,7 @@ For each unanswered human comment: | `gh issue view --json title,body,state,labels,author` | Fetch full issue metadata as JSON | | `gh issue view --json comments --jq '.comments[].body'` | Fetch all comments on an issue | | `gh issue comment --body "..."` | Post a comment on an issue | -| `gh issue edit --add-label "state:review-ready"` | Add a label to an issue | +| `gh issue edit --remove-label "agent:plan-requested" --add-label "agent:plan-ready"` | Mark a remediation plan ready for human review | ## Example Usage @@ -176,7 +180,7 @@ User says: "Review security issue #42" 3. No prior review found -- pass issue to `principal-engineer-reviewer` with security lens 4. Reviewer determines it's a legitimate XSS vulnerability in the API response handler 5. Post a comment with severity assessment and remediation plan -6. Add the `state:review-ready` label to the issue +6. If `agent:plan-requested` was present, replace it with `agent:plan-ready`; otherwise leave the direct invocation unlabeled 7. Report the finding and posted comment to the user ### Re-review with new comments diff --git a/.agents/skills/sync-agent-infra/SKILL.md b/.agents/skills/sync-agent-infra/SKILL.md index 2c87dd4182..bd01bfebde 100644 --- a/.agents/skills/sync-agent-infra/SKILL.md +++ b/.agents/skills/sync-agent-infra/SKILL.md @@ -9,8 +9,9 @@ Detect and fix drift across the agent-first infrastructure files. These files re | File | What it tracks | |------|---------------| -| `AGENTS.md` | Project identity, workflow chains, architecture overview, issue/PR conventions | +| `AGENTS.md` | Project identity, workflow chains, architecture overview, issue/PR conventions, skill maintenance pointer | | `CONTRIBUTING.md` | Skills table, workflow chains, "When to Open an Issue" guidance, skill references | +| `docs/resources/issue-lifecycle.mdx` | Human-facing issue states, roadmap decisions, and direct-versus-queued agent ownership | | `README.md` | "Built With Agents" section, "Explore with your agent" skill references | | `.github/ISSUE_TEMPLATE/bug_report.yml` | Skill name references in diagnostic guidance | | `.github/ISSUE_TEMPLATE/feature_request.yml` | Skill name references in investigation guidance | @@ -18,16 +19,43 @@ Detect and fix drift across the agent-first infrastructure files. These files re | `.github/workflows/issue-triage.yml` | Comment text referencing skills | | `.agents/skills/triage-issue/SKILL.md` | Skill name references in gate check and diagnosis steps | | `.agents/skills/openshell-cli/SKILL.md` | Companion skills table | -| `.agents/skills/build-from-issue/SKILL.md` | `state:triage-needed` label awareness | +| `.agents/skills/create-github-pr/SKILL.md` | Pre-PR agent infrastructure check | +| `.agents/skills/review-github-pr/SKILL.md` | Review-time agent infrastructure check | +| `.agents/skills/build-from-issue/SKILL.md` | Label awareness and pre-commit agent infrastructure check | +| `.claude/agents/principal-engineer-reviewer.md` | Shared review-time agent infrastructure check | ## When to Run - After adding, removing, or renaming a skill in `.agents/skills/` - After adding, removing, or renaming a crate in `crates/` - After changing workflow chain relationships between skills +- After changing which product or development areas a skill covers - After modifying issue or PR templates - Before opening a PR that touches any of the above +## Skill Maintenance Map + +Use this map when product behavior, commands, or development workflows change. It is a routing aid, not an exhaustive dependency list. Search `.agents/skills/` for the changed command, field, component, or workflow before concluding that no other skill needs an update. + +| Change area | Skills to review | +|---|---| +| CLI commands, flags, defaults, or workflows | `openshell-cli` | +| Sandbox policy schema, presets, or enforcement behavior | `generate-sandbox-policy`, `openshell-cli` | +| Supervisor middleware policy, registrations, runtime, or failure behavior | `generate-sandbox-policy`, `openshell-cli`, `debug-openshell-cluster` | +| Gateway deployment, Helm, runtime drivers, or health checks | `debug-openshell-cluster`, `helm-dev-environment` | +| Inference routing, providers, or `inference.local` behavior | `debug-inference`, `openshell-cli` | +| TUI architecture, navigation, data fetching, or UX | `tui-development` | +| Release artifacts or post-publish smoke coverage | `test-release-canary` | +| GitHub Actions workflows, required checks, or CI diagnostics | `watch-github-actions`; also `test-release-canary` for release smoke coverage | +| Gator harness, sandbox image, supervision, or model overrides | `launch-openshell-gator` | +| SBOM generation, dependency metadata, or license workflows | `sbom` | +| Issue templates, labels, contribution gates, or spike/build workflow | `triage-issue`, `create-spike`, `build-from-issue`, `create-github-issue` | +| PR template, review conventions, or vouch behavior | `create-github-pr`, `review-github-pr`, `build-from-issue` | +| Security review or remediation workflow | `review-security-issue`, `fix-security-issue` | +| RFC template, numbering, or lifecycle | `create-rfc` | +| Documentation structure, navigation, or doc-update workflow | `update-docs` | +| Skills, crates, workflow chains, issue/PR templates, or agent cross-references | `sync-agent-infra` | + ## Prerequisites You must be in the OpenShell repository root. @@ -60,7 +88,7 @@ The canonical workflow chains are defined in `AGENTS.md` under "## Workflow Chai ### Labels -The canonical label set is used by skills and templates. The key labels are: `state:agent-ready`, `state:review-ready`, `state:in-progress`, `state:pr-opened`, `state:triage-needed`, `topic:security`, `good first issue`, `spike`, and the relevant `area:*`, `topic:*`, `integration:*`, and `test:*` labels. +The canonical label set is used by skills and templates. The key labels are: `state:triage-needed`, `state:needs-info`, `state:validated`, `state:accepted`, `agent:plan-requested`, `agent:plan-ready`, `agent:implementation-requested`, `agent:in-progress`, `agent:pr-opened`, `roadmap`, `topic:security`, `good first issue`, `help wanted`, `spike`, and the relevant `area:*`, `topic:*`, `integration:*`, and `test:*` labels. The `agent:*` request labels control unattended queue pickup; they are not prerequisites when a user directly asks an agent to work on a specific issue. ## Step 2: Check Each File for Drift @@ -77,6 +105,12 @@ For each file in the table above, check for the following inconsistencies: 1. **Architecture overview** — Every crate in `crates/` must appear in the architecture table. The `python/`, `proto/`, `deploy/`, `.agents/` rows must also be present. 2. **Workflow chains** — Verify each skill named in a chain exists in `.agents/skills/`. 3. **Issue/PR conventions** — Verify referenced skills (`create-github-issue`, `create-github-pr`, `build-from-issue`) exist. +4. **Skill maintenance pointer** — Verify it still points to `sync-agent-infra` and does not duplicate the maintenance map from this skill. + +### Issue Lifecycle Documentation + +1. **`docs/resources/issue-lifecycle.mdx`** — State, roadmap, and agent-workflow meanings must match `AGENTS.md` and `CONTRIBUTING.md`. +2. **Invocation modes** — The `agent:*` request labels must control unattended queue pickup without being presented as prerequisites for a direct user request to a specific agent. ### `README.md` @@ -85,8 +119,8 @@ For each file in the table above, check for the following inconsistencies: ### Issue Templates -1. **`bug_report.yml`** — Skill names in the Agent Diagnostic guidance and checklist must exist. -2. **`feature_request.yml`** — Skill names in the Agent Investigation guidance must exist. +1. **`bug_report.yml`** — Must collect a User Story, Problem Statement, Impact / Why This Matters, Acceptance Criteria, Reproduction Steps, and Environment. Logs are optional and bug-specific; reporter diagnostics must not be required. +2. **`feature_request.yml`** — Must collect a User Story, Problem Statement, Impact / Why This Matters, Proposed Design, Acceptance Criteria, and Alternatives Considered. The design describes workflow and observable behavior without prescribing internal implementation; agent investigation is optional. 3. **`config.yml`** — Skill category descriptions in contact links should be accurate. ### Issue Triage Workflow @@ -97,9 +131,10 @@ For each file in the table above, check for the following inconsistencies: 1. **`triage-issue`** — Skills referenced in gate check and diagnosis steps must exist. 2. **`openshell-cli`** — Companion skills table entries must exist. -3. **`build-from-issue`** — Label names must match the project's label taxonomy. +3. **`build-from-issue`** — Label names must match the project's label taxonomy, and request labels must gate unattended queue pickup without blocking direct user requests. 4. **`create-spike`** — Reference to `build-from-issue` as next step must be accurate. 5. **`review-security-issue`** / **`fix-security-issue`** — Cross-references between the two must be accurate. +6. **PR creation and review checks** — The `create-github-pr`, `review-github-pr`, `build-from-issue`, and `principal-engineer-reviewer` references to `sync-agent-infra` must exist and use trigger conditions aligned with this skill. ## Step 3: Report Drift @@ -125,6 +160,7 @@ If any inconsistencies are found, report them in a structured format: ### Cross-References - : references non-existent skill - : references non-existent label