diff --git a/README.md b/README.md index 7d9dcd8..a711d09 100644 --- a/README.md +++ b/README.md @@ -17,11 +17,16 @@ harness apply -f harness.yaml # launch a sandbox Launch an interactive coding session with Claude Code or OpenCode. ```bash -harness apply --attach # local Podman with built-in harness -harness apply -f harness.yaml --attach --gateway openshift # Agent config in harness.yaml on OpenShift +harness apply --attach # built-in default agent +harness apply -f harness.yaml --attach # agent config in harness.yaml harness apply -f harness.yaml --attach --entrypoint opencode # OpenCode ``` +`harness apply` runs against whichever gateway OpenShell has provisioned and you +have selected (`openshell gateway select`) — local or cluster, the harness YAML +is identical. Provisioning the gateway is OpenShell's job, not the harness's (see +[Install](#install)). + ### One-shot tasks Run a task headlessly -- the agent executes in a sandbox and outputs results. @@ -50,13 +55,15 @@ To get results out: `--task` mode outputs to stdout, `openshell sandbox exec` pu ## Why this exists -[OpenShell](https://github.com/NVIDIA/OpenShell) provides a strict, secure sandbox runtime — deny-by-default L7 network policy, credential proxying, Landlock filesystem isolation, and inference routing. What it doesn't provide is the developer workflow layer on top: the config that wires up providers, the deployment abstraction that works the same locally and on a cluster, or the CI harness that catches breakage before developers hit it. +[OpenShell](https://github.com/NVIDIA/OpenShell) provides a strict, secure sandbox runtime — deny-by-default L7 network policy, credential proxying, Landlock filesystem isolation, and inference routing. It also provisions the gateway itself (the local installer, or `helm install openshell` on a cluster). What it doesn't provide is the developer workflow layer on top: the config that wires up providers, the declarative reconciliation that makes a gateway match your intent, or the CI harness that catches breakage before developers hit it. + +Without a shared harness layer, every team building on OpenShell independently solves the same problems — writing shell scripts to register providers, hand-rolling container images, re-deriving inference routing. The configs diverge, the security posture varies, and nobody catches regressions until something breaks in production. -Without a shared harness layer, every team building on OpenShell independently solves the same problems — writing shell scripts to register providers, hand-rolling container images, maintaining separate deployment procedures per environment. The configs diverge, the security posture varies, and nobody catches regressions until something breaks in production. +**The design boundary**: managing a gateway is OpenShell's problem; the harness is a declarative setup/run layer with zero compute-backend opinion. It never provisions or tears down a gateway — it declares providers, inference, and policy against one OpenShell already stood up, and runs agents in it. That keeps the harness YAML portable: the same file targets a local gateway or a cluster gateway with no target field to change. -**The core design constraint**: if the developer harness isn't running and live-tested in CI, the developer experience can't be maintained. OpenShell, agent CLIs, and provider APIs all change frequently — often multiple times per week. A harness that works today and isn't continuously validated will silently break. harness-openshell runs the full lifecycle (deploy gateway → register providers → create sandbox → run task → tear down) in CI on every change, across three deployment targets: local Podman, Kind, and OpenShift. +**The core design constraint**: if the developer harness isn't running and live-tested in CI, the developer experience can't be maintained. OpenShell, agent CLIs, and provider APIs all change frequently — often multiple times per week. A harness that works today and isn't continuously validated will silently break. harness-openshell runs the workflow (register providers → reconcile inference → create sandbox → run task) in CI on every change, against gateways provisioned three ways: local Podman, Kind, and OpenShift. -**The path from local to automated**: a developer runs `harness apply --attach` for interactive work. When the workflow is ready for CI, they change `--attach` to `--task @skill.md` and `gateway: local-container` to `gateway: openshift`. Everything else stays the same. No rewriting, no separate deployment tooling. The harness YAML is the artifact — sharable, versionable, forkable. +**The path from local to automated**: a developer runs `harness apply --attach` for interactive work. When the workflow is ready for CI, they change `--attach` to `--task @skill.md` and select a cluster gateway instead of the local one. The harness YAML stays the same. No rewriting. The harness YAML is the artifact — sharable, versionable, forkable. OpenShell's upstream direction is toward a [Kubernetes Operator](https://github.com/NVIDIA/OpenShell/issues/1719) where providers and sandboxes become CRDs and the gateway narrows to data-plane only. The harness explores what the workflow layer looks like above that with a developer mindset from local machine to cluster. @@ -135,16 +142,17 @@ This inherits all four providers and inference routing from `agent-default.yaml` ## How It Works ``` +(OpenShell has already provisioned the gateway; you selected it) harness apply -f config.yaml | - +-> Deploy gateway (Podman container or K8s StatefulSet) +-> Register providers (credentials from host env) + +-> Reconcile inference routing to match the config +-> Upload payloads (CLAUDE.md, MCP config, skills) +-> Create sandbox (isolated container, deny-by-default network) +-> Run task (agent executes, outputs results) ``` -OpenShell provides the runtime isolation. The harness provides the workflow. +OpenShell provisions the gateway and provides the runtime isolation. The harness provides the workflow. For runtime operations and policy management, use openshell directly: ```bash @@ -174,10 +182,11 @@ Install a bare `brew install openshell` off the tap and you get whatever version the formula defaults to — usually behind. `make openshell` runs the upstream `install.sh` at the pinned version instead, so local matches CI exactly. -The installer starts the gateway service; register it once: +The installer starts the gateway service; register and select it once: ```bash openshell gateway add https://127.0.0.1:17670 --local --name openshell +openshell gateway select openshell ``` If you need to restart the service later: `brew services restart openshell` @@ -185,6 +194,28 @@ If you need to restart the service later: `brew services restart openshell` Or build the harness from source: `make cli` +### On a cluster + +Provisioning a cluster gateway is OpenShell's job too — the harness has no +`deploy` command. Install the chart, then register and select the gateway: + +```bash +helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart +openshell gateway add https:// --name my-cluster +openshell gateway select my-cluster +harness apply -f harness.yaml # same YAML, cluster gateway +``` + +Tear the gateway down with `helm uninstall openshell` and +`openshell gateway remove my-cluster`. The harness `delete` command removes +sandboxes; add `--providers` (or `--all`) to remove providers too. It never +removes the gateway. + +> **Migration:** `harness deploy`, `harness teardown`, `harness status`, and +> `delete --k8s` are removed. Provision the gateway with OpenShell (the +> `openshell` installer or `helm install openshell`); the harness declares +> providers/inference/policy and runs agents against it. + ## Reference ### Commands @@ -199,7 +230,6 @@ Or build the harness from source: `make cli` | `harness apply --attach` | Interactive TTY mode | | `harness apply --dry-run` | Validate without deploying | | `harness apply -o yaml` | Output resolved config | -| `harness deploy ` | Deploy gateway only | | `harness get agents\|providers\|gateways` | List resources | | `harness describe ` | Sandbox details | | `harness delete [--all]` | Tear down | @@ -223,7 +253,6 @@ Each provider discovers credentials from the host. Missing providers are skipped |------|---------| | `profiles/agent-*.yaml` | Agent configs | | `profiles/providers/` | Provider profiles (imported to gateway) | -| `profiles/gateways/*.yaml` | Gateway profiles per target | | `profiles/images/sandbox-default/` | Sandbox image defaults (overridable via payloads) | ## Testing @@ -233,17 +262,17 @@ Tested on macOS (arm64) with Podman. Linux support is expected but not yet valid ```bash make test # vet + unit tests (16 packages) make lint # golangci-lint -make test-suite # config parsing (33 tests, no gateway needed) +make test-suite # config parsing (32 tests, no gateway needed) make test-local # full e2e on local Podman (22 tests) make test-kind # self-contained kind cluster lifecycle make test-remote # full e2e on OCP (needs KUBECONFIG) ``` -`test-local` is the primary validation target. It deploys the gateway, registers all 4 providers, creates sandboxes, verifies exec/env/GWS token resolution/MCP config/Claude inference, tests missing-provider recovery, and tears down. +`test-local` is the primary validation target. It provisions a gateway via the OpenShell installer, registers all 4 providers, creates sandboxes, verifies exec/env/GWS token resolution/MCP config/Claude inference, tests missing-provider recovery, and tears down the sandboxes and providers it created (the gateway is OpenShell's to remove). -`test-kind` creates its own kind cluster, builds and loads the sandbox image, runs the full flow, and deletes the cluster on exit. Use `KEEP=1` to keep the cluster for debugging. +`test-kind` creates its own kind cluster, `helm install`s OpenShell, builds and loads the sandbox image, runs the full flow, and deletes the cluster on exit. Use `KEEP=1` to keep the cluster for debugging. -`test-remote` requires `KUBECONFIG` pointing at an OCP cluster and pushes the image automatically. Use `--reuse-gateway` to skip deploy/teardown when iterating. +`test-remote` requires `KUBECONFIG` pointing at an OCP cluster and pushes the image automatically. Use `--reuse-gateway` to skip gateway provisioning/teardown when iterating. Each integration target builds (and pushes, for remote) the sandbox image automatically. diff --git a/SPEC.md b/SPEC.md index ad6bce7..0fa6a28 100644 --- a/SPEC.md +++ b/SPEC.md @@ -4,10 +4,12 @@ Behavior specification for the OpenShell Harness CLI. ## Overview -The harness deploys and manages AI agent sandboxes on three targets: -- **local-container** -- Podman containers via a local OpenShell gateway -- **helm** -- Kubernetes pods via a k8s cluster (NodePort access) -- **openshift** -- Kubernetes pods via an OpenShift-hosted OpenShell gateway (Route access) +The harness declares providers, inference routing, and policy, then creates and +runs AI agent sandboxes against a gateway **OpenShell has already provisioned**. +Provisioning a gateway (local installer, or `helm install openshell` on a +cluster) is OpenShell's job — the harness has zero compute-backend opinion and +never deploys or tears down a gateway. It runs against whichever gateway is +selected (`openshell gateway select`), local or cluster, from the same YAML. Each sandbox is an isolated container running an agent entrypoint (e.g. Claude Code, OpenCode, or Codex; `bash` or any binary on PATH also works), with credential providers, network policies, and a rendered payload (`task.md` and a `bin/` directory). @@ -37,7 +39,7 @@ env: Fields: - `name` (required) -- sandbox name, used for `openshell sandbox connect` -- `base_agent` -- name of a base agent config to inherit from (e.g., `default` resolves `agent-default.yaml`). Providers, env, and payloads are merged additively; scalar fields (entrypoint, gateway, repo, task, image, policy) from the overlay win when non-empty. +- `base_agent` -- name of a base agent config to inherit from (e.g., `default` resolves `agent-default.yaml`). Providers, env, and payloads are merged additively; scalar fields (entrypoint, repo, task, image, policy) from the overlay win when non-empty. - `image` -- container image for the sandbox (default: version-matched from `quay.io/rcochran/openshell`, override with `HARNESS_OS_IMAGE` env) - `entrypoint` -- command to run (default: `claude`). Supports `claude`, `codex`, `opencode`, `bash`, or any binary on PATH. - `tty` -- enable TTY (default: true) @@ -50,13 +52,15 @@ Fields: - `env` -- additional environment variables injected via `--env` on sandbox create (empty values read from host env) - `include` -- extra files to include in the payload - `policy` -- path to a network policy YAML -- `gateway` -- target gateway name (overrides active gateway) + +The agent config names no gateway: `harness apply` runs against the gateway +OpenShell has provisioned and you have selected (`openshell gateway select`). Provider profiles live in `profiles/providers/`. These are imported to the gateway during provider registration. ### Multi-document harness YAML -Agent configs support multi-document YAML (`---` separated) where provider, gateway, and policy definitions are co-located in one file: +Agent configs support multi-document YAML (`---` separated) where provider and policy definitions are co-located in one file: ```yaml --- @@ -71,35 +75,37 @@ name: github type: github credentials: [GITHUB_TOKEN] --- -kind: gateway -name: local-container -type: local +kind: policy +network_policies: + github_git: + endpoints: + - host: github.com + port: 443 ``` Documents are dispatched by `kind` field. No `kind` field = agent (backwards compatible). Definitions in the harness file take priority over the `profiles/` tree. ## CLI -### `harness apply [-f FILE] [--agent NAME] [--gateway NAME] [--gateway-profile FILE] [--name SANDBOX] [--attach] [--setup-only] [--dry-run] [-o yaml|json]` +### `harness apply [-f FILE] [--agent NAME] [--name SANDBOX] [--task TEXT|@FILE] [--entrypoint CMD] [--attach] [--setup-only] [--dry-run] [-o yaml|json]` -Primary command. Resolves an agent config, deploys the gateway and providers, creates a sandbox. +Primary command. Resolves an agent config, reconciles providers and inference on the selected gateway, creates a sandbox. It never provisions a gateway — one must already be provisioned by OpenShell and selected. 1. **Parse agent config** -- resolve `agent-.yaml` from harness directory (default: `default`). `-f` overrides with a direct file path. Falls back to embedded `agent-basic.yaml` when `agent-default.yaml` is not found on disk. 2. **Check output mode** -- if `-o yaml` or `-o json`, render the fully resolved config and exit. No gateway interaction needed. 3. **Check version** -- warn if openshell CLI is below v0.0.110. -4. **Resolve gateway** -- `--gateway` selects a profile by name; `--gateway-profile` loads from a file path. Default: `local-container`. `OPENSHELL_GATEWAY` env var is used as fallback. +4. **Require an active gateway** -- resolve the gateway from `openshell`'s active selection (`OPENSHELL_GATEWAY` env var overrides). Error up front if none is selected; `upLocal` additionally preflights that the gateway is reachable before touching providers or creating a sandbox. 5. **Dry-run check** -- if `--dry-run`, validate each step (gateway reachable, providers resolvable, env vars resolved, image available) and exit with pass/fail report. -6. **Ensure gateway** -- deploy if needed (local: Podman, remote: Helm to K8s/OCP). -7. **Ensure providers** -- auto-register missing providers. Three registration flows: +6. **Ensure providers** -- auto-register missing providers. Three registration flows: - **Standard** (`--from-existing`): GitHub, Atlassian -- OpenShell discovers credentials from local env. - **ADC** (`--from-gcloud-adc`): Vertex AI -- reads ADC file, configures inference routing. - **Custom**: GWS -- multi-step OAuth refresh flow. -8. **Render payload** -- `task.md` (if set) and a `bin/` directory. The in-sandbox command is built by the agent adapter (`internal/agent/adapter.go`) as a `bash -lc` invocation (PATH setup, entrypoint validation via `command -v`), not a `run.sh` file. Task dispatch depends on mode and entrypoint: headless (default) uses `opencode run "$(cat task.md)"` for OpenCode and `--print "$(cat task.md)"` for claude/codex/custom entrypoints; interactive (`--attach`) uses `-p "$(cat task.md)"`. -9. **Create sandbox** -- `openshell sandbox create` with `--env` (env vars), `--upload` (payload), and startup command. Retry up to 5 times. +7. **Render payload** -- `task.md` (if set) and a `bin/` directory. The in-sandbox command is built by the agent adapter (`internal/agent/adapter.go`) as a `bash -lc` invocation (PATH setup, entrypoint validation via `command -v`), not a `run.sh` file. Task dispatch depends on mode and entrypoint: headless (default) uses `opencode run "$(cat task.md)"` for OpenCode and `--print "$(cat task.md)"` for claude/codex/custom entrypoints; interactive (`--attach`) uses `-p "$(cat task.md)"`. +8. **Create sandbox** -- `openshell sandbox create` with `--env` (env vars), `--upload` (payload), and startup command. Retry up to 5 times. Default is non-interactive (headless). Use `--attach` for TTY mode. -`--setup-only` deploys the gateway and reconciles providers/inference, then stops before creating a sandbox or running the agent. +`--setup-only` reconciles providers/inference on the selected gateway, then stops before creating a sandbox or running the agent. ### `harness get [-o table|json|yaml]` @@ -117,21 +123,17 @@ These are convenience wrappers. For full details, use `openshell sandbox list`, Show detailed status for a specific sandbox: phase, active gateway, and registered providers. -### `harness delete [NAME...] [--all] [--providers] [--k8s]` +### `harness delete [NAME...] [--all] [--providers]` -Delete sandboxes by name, or use flags for bulk operations. `--all` deletes sandboxes, providers, and k8s resources. Reuses the same teardown functions as the old `teardown` command. +Delete sandboxes by name, or use flags for bulk operations. `--all` deletes sandboxes and providers. It never removes the gateway: cluster teardown is `helm uninstall openshell` plus `openshell gateway remove`. ### `harness init [-o FILE] [--force] [--non-interactive]` -Generate a `harness.yaml` config file. Interactive by default (prompts for entrypoint, providers, and gateway target); `--non-interactive` writes the embedded default. Writes to `harness.yaml` unless `-o` overrides the path. +Generate a `harness.yaml` config file. Interactive by default (prompts for entrypoint and providers); `--non-interactive` writes the embedded default. Writes to `harness.yaml` unless `-o` overrides the path. The generated config names no gateway — select one with `openshell gateway select`. ### `harness doctor [-f FILE] [--agent NAME] [--gateway NAME] [-o table|json|yaml]` -Validate the environment for a configured sandbox. Phase 1 (offline) checks the openshell binary, target dependencies, and provider credentials without a running gateway; Phase 2 (online) checks provider registration when the gateway is reachable. - -### `harness deploy ` - -Deploy or verify the gateway for a target. Reads `profiles/gateways/.yaml`. +Validate the environment for a configured sandbox. Phase 1 (offline) checks the openshell binary and provider credentials without a running gateway; Phase 2 (online) checks provider registration when the gateway is reachable. ### `harness plan -f FILE [--gateway NAME] [-o table|json|yaml]` @@ -139,16 +141,7 @@ Read-only reconciliation plan. Shows the actions `harness apply` would take with ### `harness migrate -f FILE [-o FILE]` -Convert a legacy v1 harness config to the v1alpha1 format. The input YAML is normalized and written as v1alpha1 to stdout (or `-o FILE`). Fields with no v1alpha1 home (`task`, `include`, inline policy documents, unresolved `base_agent`) are reported as warnings on stderr. - -### Deprecated Aliases - -These commands still work but will be removed in a future release: - -| Old command | Replacement | Notes | -|-------------|-------------|-------| -| `harness teardown` | `harness delete` | Same flags: `--sandboxes`, `--providers`, `--k8s` | -| `harness status` | `harness get agents` | | +Convert a legacy v1 harness config to the v1alpha1 format. The input YAML is normalized and written as v1alpha1 to stdout (or `-o FILE`). Fields with no v1alpha1 home (`task`, `include`, inline policy documents, unresolved `base_agent`) are reported as warnings on stderr. The legacy `gateway:` field named a deploy profile, a concept the harness no longer owns; it is dropped, leaving `spec.target.gateway` empty. The active gateway is chosen outside the config — with `openshell gateway select` or `$OPENSHELL_GATEWAY` — not by re-adding a field to the YAML. ## Config Files @@ -156,7 +149,6 @@ These commands still work but will be removed in a future release: |------|---------| | `profiles/agent-*.yaml` | Agent config: image, entrypoint, providers, env, task | | `profiles/providers/` | OpenShell provider profile YAMLs | -| `profiles/gateways/*.yaml` | Gateway profiles: deployment target config with inline Helm values | | `profiles/images/sandbox-default/Dockerfile` | Sandbox image: OpenShell base + MCP servers + CLI tools | | `profiles/images/sandbox-default/CLAUDE.md` | Claude Code project instructions for sandbox | | `profiles/images/sandbox-default/claude.json` | Claude Code settings | @@ -190,14 +182,9 @@ Harness-specific variables use the `HARNESS_OS_` prefix. OpenShell runtime varia |----------|---------| | `HARNESS_OS_DIR` | Override harness directory detection | | `HARNESS_OS_IMAGE` | Override sandbox image (dev/CI builds) | -| `HARNESS_OS_PULL_SECRET` | Image pull secret name passed to Helm install | -| `HARNESS_OS_SANDBOX_PULL_SECRET` | Sandbox image pull secret name passed to Helm install | | `OPENSHELL_CLI` | Override openshell binary path | -| `OPENSHELL_GATEWAY` | Override gateway name (used by apply, plugin-compatible) | -| `OPENSHELL_NAMESPACE` | Override K8s namespace (default: `openshell`) | +| `OPENSHELL_GATEWAY` | Override the active gateway name (used by apply, plugin-compatible) | | `OPENSHELL_MODEL` | Inference model for provider registration (default: `claude-sonnet-4-6`) | -| `OPENSHELL_CHART_VERSION` | Override Helm chart version (beats `gateway.yaml`) | -| `KUBECONFIG` | K8s cluster config for remote targets | ## Payload diff --git a/TODO.md b/TODO.md index 6f395d5..e22e5de 100644 --- a/TODO.md +++ b/TODO.md @@ -3,14 +3,14 @@ ## Next up ### `harness init` [DONE] -- [x] Generate a harness.yaml with interactive prompts (entrypoint, providers, gateway) +- [x] Generate a harness.yaml with interactive prompts (entrypoint, providers) - [x] Discover providers from `openshell provider list-profiles` - [x] Print next steps ("run `harness doctor` then `harness apply`") - [x] `--non-interactive`, `--force`, `--output` flags ### `harness doctor` [DONE] - [x] Check openshell installed and version -- [x] Check target-specific deps (podman/docker, kubectl, kind, kubeconfig) +- [x] Check gateway reachability online (target-specific infra checks removed in PR7b — the harness no longer provisions, so it has no compute-backend deps to check) - [x] Check provider credentials via `openshell provider profile export` - [x] Online phase: check provider registration if gateway reachable - [x] `-o table|json|yaml` output @@ -24,16 +24,17 @@ - [x] `harness apply` with `--dry-run`, `-o yaml|json`, `--attach`, `-f`, `--task`, `--entrypoint` - [x] `harness get agents|providers|gateways` with `-o table|json|yaml` - [x] `harness describe ` with `-o table|json|yaml` -- [x] `harness delete ` with `--all`, `--sandboxes`, `--providers`, `--k8s` -- [x] `harness deploy [local|ocp|kind]` +- [x] `harness delete ` with `--all`, `--sandboxes`, `--providers` - [x] Headless task mode: `--task "text"` or `--task @file` runs agent with `--print` - [x] `kind: policy` applied via `openshell policy set` after sandbox creation -- [x] `teardown` and `status` as hidden deprecated aliases - [x] `up`, `create`, `render`, `start`, `stop` removed +- [x] `deploy`, `teardown`, `status`, and `delete --k8s` removed (PR7b): the + harness no longer provisions gateways — provision with OpenShell + (`openshell` installer or `helm install openshell`) ## Agent Config [DONE] -- [x] Multi-document harness YAML (`kind: agent/provider/gateway/payload/policy`) +- [x] Multi-document harness YAML (`kind: agent/provider/payload/policy`; `kind: gateway` still parses but is inert after PR7b) - [x] `kind: payload` with `sandbox_path`/`local_path`/`content` + multi-upload - [x] Agent-level `payloads:` list merged with document-level payloads - [x] `kind: config` kept as silent alias for backwards compat diff --git a/cmd/apply.go b/cmd/apply.go index 863dcb5..ec9121f 100644 --- a/cmd/apply.go +++ b/cmd/apply.go @@ -19,8 +19,6 @@ func NewApplyCmd(harnessDir, cli string, newClient openshell.Factory) *cobra.Com var ( file string agentName string - gatewayName string - gatewayProfile string sandboxName string task string entrypoint string @@ -33,13 +31,12 @@ func NewApplyCmd(harnessDir, cli string, newClient openshell.Factory) *cobra.Com cmd := &cobra.Command{ Use: "apply [flags]", Short: "Apply an agent configuration to create a sandbox", - Long: `Resolve an agent config against the profiles directory and running gateway, -then deploy a sandbox. Use --dry-run to validate without deploying, or + Long: `Resolve an agent config against the profiles directory and the active +OpenShell gateway, then create a sandbox. Provision the gateway with OpenShell +first (installer or 'helm install openshell') and select it with +'openshell gateway select'. Use --dry-run to validate without creating, or -o yaml to output the fully resolved configuration.`, RunE: func(cmd *cobra.Command, args []string) error { - if gatewayName != "" && gatewayProfile != "" { - return fmt.Errorf("--gateway and --gateway-profile are mutually exclusive") - } if len(args) > 0 && sandboxName == "" { sandboxName = args[0] } @@ -95,46 +92,35 @@ then deploy a sandbox. Use --dry-run to validate without deploying, or gw := gateway.New(cli) if err := gw.CheckMinVersion(gateway.MinOpenShellVersion); err != nil { - // A CLI that is definitively too old will fail deployment later - // with far less context, so refuse up front. If we merely could - // not read/parse the version, warn and proceed — the CLI may - // still be usable and we don't want to block on a format change. + // A CLI that is definitively too old will fail later with far + // less context, so refuse up front. If we merely could not + // read/parse the version, warn and proceed — the CLI may still + // be usable and we don't want to block on a format change. if errors.Is(err, gateway.ErrVersionBelowMinimum) { return fmt.Errorf("incompatible openshell CLI: %w", err) } status.Warn(fmt.Sprintf("OpenShell version: %v", err)) } - // Resolve gateway - var gwCfg *gateway.GatewayConfig - gwTarget := gatewayName - if gatewayProfile != "" { - gwCfg, err = resolveGatewayConfigFromFile(gatewayProfile) - if err != nil { - return err - } - gwTarget = gwCfg.Gateway.Type - } else { - if gwTarget == "" { - if agentCfg.Gateway != "" { - gwTarget = agentCfg.Gateway - } else { - gwTarget = "local-container" - } - } - gwCfg, _ = resolveGatewayConfigWithHarness(harnessDir, gwTarget, harness) + // The harness runs against a gateway OpenShell already provisioned; + // it never provisions one. Resolve the target once here — this is the + // single owner of apply's gateway selection ($OPENSHELL_GATEWAY > + // active marker) — and thread it through so reconcile and + // sandbox-create act on the same gateway. Fail clearly here if none is + // selected instead of deep in reconcile/run. + target, err := resolveApplyTarget(gw) + if err != nil { + return err } - isRemote := gwCfg != nil && !gwCfg.IsLocal() if dryRun { - return dryRunApply(gw, agentCfg, gwTarget, isRemote) + return dryRunApply(gw, target, agentCfg) } return upLocal(upLocalOpts{ harnessDir: harnessDir, gw: gw, - gwCfg: gwCfg, - ensureLocal: !isRemote, + target: target, agentCfg: agentCfg, agentPath: agentPath, sandboxName: sandboxName, @@ -149,14 +135,12 @@ then deploy a sandbox. Use --dry-run to validate without deploying, or cmd.Flags().StringVarP(&file, "file", "f", "", "Path to harness/agent YAML file") cmd.Flags().StringVar(&agentName, "agent", "default", "Agent config name (from profiles/)") - cmd.Flags().StringVar(&gatewayName, "gateway", envOr("OPENSHELL_GATEWAY", ""), "Gateway profile name") - cmd.Flags().StringVar(&gatewayProfile, "gateway-profile", "", "Path to gateway profile YAML") cmd.Flags().StringVar(&sandboxName, "name", "", "Sandbox name (overrides agent config)") cmd.Flags().StringVar(&task, "task", "", "Task to pass to the agent (inline text or @filepath)") cmd.Flags().StringVar(&entrypoint, "entrypoint", "", "Override agent entrypoint (claude, opencode, bash)") cmd.Flags().BoolVar(&attach, "attach", false, "Attach TTY after creation (interactive mode)") cmd.Flags().BoolVar(&dryRun, "dry-run", false, "Validate configuration without deploying") - cmd.Flags().BoolVar(&setupOnly, "setup-only", false, "Deploy the gateway and reconcile providers/inference, but do not create a sandbox or run the agent") + cmd.Flags().BoolVar(&setupOnly, "setup-only", false, "Reconcile providers/inference on the active gateway, but do not create a sandbox or run the agent") cmd.Flags().StringVarP(&output, "output", "o", "", "Output format: yaml or json") return cmd @@ -165,16 +149,6 @@ then deploy a sandbox. Use --dry-run to validate without deploying, or func renderOutput(harnessDir string, h *agent.Harness, format string) error { builtinProviders := loadProviderProfiles(harnessDir) - gwName := h.Agent.Gateway - if gwName == "" { - gwName = "local-container" - } - if len(h.Gateways) == 0 { - if gwData := loadGatewayProfile(harnessDir, gwName); gwData != nil { - h.Gateways[gwName] = gwData - } - } - switch format { case "yaml": out, err := agent.RenderHarness(h, builtinProviders) @@ -206,7 +180,7 @@ func mapKeys(m map[string][]byte) []string { return keys } -func dryRunApply(gw gateway.Gateway, agentCfg *agent.AgentConfig, gwTarget string, isRemote bool) error { +func dryRunApply(gw gateway.Gateway, target openshell.Target, agentCfg *agent.AgentConfig) error { status.Header("Dry Run") allPass := true @@ -215,15 +189,14 @@ func dryRunApply(gw gateway.Gateway, agentCfg *agent.AgentConfig, gwTarget strin image := resolveSandboxImage(agentCfg.Image) status.OKf("image: %s", image) - if isRemote { - if gw.InferenceGet() != nil { - status.Failf("gateway: %s (not reachable)", gwTarget) - allPass = false - } else { - status.OKf("gateway: %s (reachable)", gwTarget) - } + // Report the resolved target — the same gateway apply will act on — not the + // raw active marker, so a $OPENSHELL_GATEWAY-targeted dry run matches apply. + gwName := target.Gateway + if gw.InferenceGet() != nil { + status.Failf("gateway: %s (not reachable)", gwName) + allPass = false } else { - status.OKf("gateway: %s (local)", gwTarget) + status.OKf("gateway: %s (reachable)", gwName) } for _, p := range agentCfg.Providers { diff --git a/cmd/delete.go b/cmd/delete.go index c8ee19e..a9aa5c6 100644 --- a/cmd/delete.go +++ b/cmd/delete.go @@ -8,52 +8,57 @@ import ( "github.com/spf13/cobra" "github.com/stackrox/harness-openshell/internal/gateway" - "github.com/stackrox/harness-openshell/internal/k8s" "github.com/stackrox/harness-openshell/internal/openshell" "github.com/stackrox/harness-openshell/internal/status" ) -func NewDeleteCmd(harnessDir, cli string, newClient openshell.Factory) *cobra.Command { +func NewDeleteCmd(gw gateway.Gateway, newClient openshell.Factory) *cobra.Command { var ( all bool sandboxes bool providers bool - k8sFlag bool ) var gatewayName, workspace *string cmd := &cobra.Command{ - Use: "delete [NAME...] [--all] [--providers] [--k8s]", - Short: "Delete sandboxes, providers, or k8s resources", + Use: "delete [NAME...] [--all] [--sandboxes] [--providers]", + Short: "Delete sandboxes or providers", Long: `Delete specific sandboxes by name, or use flags for bulk operations. Examples: harness delete my-sandbox Delete a specific sandbox harness delete agent test Delete multiple sandboxes - harness delete --all Delete all sandboxes, providers, and k8s resources - harness delete --providers Delete all providers (no running sandboxes allowed) - harness delete --k8s Delete k8s resources (helm, namespace, SCCs)`, + harness delete --all Delete all sandboxes and providers + harness delete --sandboxes Delete all sandboxes + harness delete --providers Delete all providers (no running sandboxes allowed)`, RunE: func(cmd *cobra.Command, args []string) error { - if len(args) == 0 && !all && !sandboxes && !providers && !k8sFlag { - return fmt.Errorf("specify sandbox name(s) or use --all, --sandboxes, --providers, --k8s") + if len(args) == 0 && !all && !sandboxes && !providers { + return fmt.Errorf("specify sandbox name(s) or use --all, --sandboxes, --providers") } ctx := cmd.Context() target := openshell.ResolveTarget(*gatewayName, *workspace, "", "", os.Getenv) + // Fall back to the CLI's active-gateway marker (set by `openshell + // gateway select`) when neither --gateway nor $OPENSHELL_GATEWAY + // pins one — the same selection apply runs against. An empty target + // does NOT mean "the active gateway": ResolveTarget never consults + // the marker, so without this delete would ignore a selected gateway + // entirely and either error or silently sweep nothing. + if target.Gateway == "" { + target.Gateway = gw.ActiveGateway() + } + // The harness only ever acts on an already-selected gateway. Without + // one there is nothing to delete from, and the bulk sweeps would + // otherwise silently skip and still report success. + if target.Gateway == "" { + return fmt.Errorf("no active openshell gateway — run 'openshell gateway select ' first (provision one with the OpenShell installer or 'helm install openshell')") + } - // The --k8s branch is CLI/kubectl-backed and needs no SDK client; - // open (and dial) one only when a sandbox/provider path will use it, - // so `delete --k8s` still works when the OpenShell API is down. - needsSDK := len(args) > 0 || all || sandboxes || providers - var client openshell.Client - if needsSDK { - var err error - client, err = newClient(ctx, target) - if err != nil { - return fmt.Errorf("create OpenShell client: %w", err) - } - defer client.Close() + client, err := newClient(ctx, target) + if err != nil { + return fmt.Errorf("create OpenShell client: %w", err) } + defer client.Close() // Targeted sandbox deletion if len(args) > 0 { @@ -64,63 +69,41 @@ Examples: status.OKf("Deleted sandbox %s", name) } } - if !all && !providers && !k8sFlag { + if !all && !providers { return nil } } - if target.Gateway != "" { - status.Infof("Active gateway: %s", target.Gateway) - } else { - status.Info("Active gateway: none") - } + status.Infof("Active gateway: %s", target.Gateway) fmt.Println() if all || sandboxes { - deleteSandboxesSDK(ctx, client, target.Gateway) + deleteSandboxesSDK(ctx, client) } if all || providers { - if err := deleteProvidersSDK(ctx, client, target.Gateway); err != nil { + if err := deleteProvidersSDK(ctx, client); err != nil { return err } } - if all || k8sFlag { - // internal/gateway residual: the --k8s path stays CLI-backed - // until PR7b retires the legacy bridge. This is the only - // sanctioned use of internal/gateway and internal/k8s in delete. - gw := gateway.New(cli) - ns := k8s.DefaultNamespace() - gwCfg := resolveFirstRemoteGateway(harnessDir) - teardownK8s(gw, gwCfg, k8s.New("", ns), k8s.New("", "")) - } status.Done("Done.") return nil }, } - cmd.Flags().BoolVar(&all, "all", false, "Delete all sandboxes, providers, and k8s resources") + cmd.Flags().BoolVar(&all, "all", false, "Delete all sandboxes and providers") cmd.Flags().BoolVar(&sandboxes, "sandboxes", false, "Delete all sandboxes") cmd.Flags().BoolVar(&providers, "providers", false, "Delete all providers") - cmd.Flags().BoolVar(&k8sFlag, "k8s", false, "Delete k8s resources") gatewayName, workspace = registerTargetFlags(cmd) return cmd } // deleteSandboxesSDK sweeps every sandbox in the target workspace over the -// OpenShell SDK. It is delete's own SDK-backed sweep, intentionally mirroring -// teardownSandboxes (cmd/teardown.go) on a different backing; the duplication is -// a short-lived seam removed in PR7b when the CLI helper and teardown command -// are retired, leaving this the single owner of the sweep. -func deleteSandboxesSDK(ctx context.Context, client openshell.Client, activeGW string) { +// OpenShell SDK. It is the sole owner of the bulk sandbox sweep. The caller has +// already resolved a non-empty gateway, so there is no no-gateway case here. +func deleteSandboxesSDK(ctx context.Context, client openshell.Client) { status.Section("Sandboxes") - if activeGW == "" { - status.Info("No active gateway, skipping") - fmt.Println() - return - } - sandboxes, err := client.Sandboxes(ctx) if err != nil { status.Fail(fmt.Sprintf("could not list sandboxes: %v", err)) @@ -140,19 +123,12 @@ func deleteSandboxesSDK(ctx context.Context, client openshell.Client, activeGW s fmt.Println() } -// deleteProvidersSDK sweeps every provider over the SDK, preserving the -// running-sandbox guard from teardownProviders (cmd/teardown.go): providers are -// refused while any sandbox is still up, with one brief retry to absorb a -// mid-deletion race. Like deleteSandboxesSDK this is a short-lived duplicate of -// the CLI helper, collapsed to the single owner in PR7b. -func deleteProvidersSDK(ctx context.Context, client openshell.Client, activeGW string) error { +// deleteProvidersSDK sweeps every provider over the SDK. Providers are refused +// while any sandbox is still up, with one brief retry to absorb a mid-deletion +// race. It is the sole owner of the bulk provider sweep. The caller has already +// resolved a non-empty gateway, so there is no no-gateway case here. +func deleteProvidersSDK(ctx context.Context, client openshell.Client) error { status.Section("Providers") - if activeGW == "" { - status.Info("No active gateway, skipping") - fmt.Println() - return nil - } - remaining, err := client.Sandboxes(ctx) if err != nil { return fmt.Errorf("could not check for running sandboxes: %w", err) diff --git a/cmd/delete_test.go b/cmd/delete_test.go index bbaf44d..3b60fdb 100644 --- a/cmd/delete_test.go +++ b/cmd/delete_test.go @@ -45,8 +45,8 @@ func TestDeleteTargeted(t *testing.T) { fc.AddSandbox("default", &types.Sandbox{Name: "agent-a", Status: types.SandboxStatus{Phase: types.SandboxReady}}) fc.AddSandbox("default", &types.Sandbox{Name: "agent-b", Status: types.SandboxStatus{Phase: types.SandboxReady}}) - cmd := NewDeleteCmd("", "", keepOpenFactory(client)) - cmd.SetArgs([]string{"agent-a"}) + cmd := NewDeleteCmd(&mockGW{}, keepOpenFactory(client)) + cmd.SetArgs([]string{"agent-a", "--gateway", "prod"}) if _, err := captureStdout(t, cmd.Execute); err != nil { t.Fatalf("delete agent-a: %v", err) } @@ -62,7 +62,7 @@ func TestDeleteSandboxesSweep(t *testing.T) { fc.AddSandbox("default", &types.Sandbox{Name: "agent-a", Status: types.SandboxStatus{Phase: types.SandboxReady}}) fc.AddSandbox("default", &types.Sandbox{Name: "agent-b", Status: types.SandboxStatus{Phase: types.SandboxReady}}) - cmd := NewDeleteCmd("", "", keepOpenFactory(client)) + cmd := NewDeleteCmd(&mockGW{}, keepOpenFactory(client)) cmd.SetArgs([]string{"--sandboxes", "--gateway", "prod"}) if _, err := captureStdout(t, cmd.Execute); err != nil { t.Fatalf("delete --sandboxes: %v", err) @@ -78,7 +78,7 @@ func TestDeleteProvidersGuard(t *testing.T) { fc.AddSandbox("default", &types.Sandbox{Name: "agent-a", Status: types.SandboxStatus{Phase: types.SandboxReady}}) fc.AddProvider("default", &types.Provider{Name: "github", Type: "github"}) - cmd := NewDeleteCmd("", "", keepOpenFactory(client)) + cmd := NewDeleteCmd(&mockGW{}, keepOpenFactory(client)) cmd.SetArgs([]string{"--providers", "--gateway", "prod"}) _, err := captureStdout(t, cmd.Execute) if err == nil { @@ -94,17 +94,55 @@ func TestDeleteProvidersGuard(t *testing.T) { } } -// Note: `delete --k8s`-only skipping the SDK client (CodeRabbit finding) is not -// unit-tested — the --k8s path invokes the real, non-injectable teardownK8s, -// which shells out to the ambient kubeconfig and would destructively act on a -// live cluster. The gating (`needsSDK`) is a simple guard in delete.go. +// Bulk deletion with no gateway resolved must fail loudly rather than skip both +// sweeps and report success — otherwise sandboxes/providers silently survive. +func TestDeleteBulkNoGatewayErrors(t *testing.T) { + t.Setenv("OPENSHELL_GATEWAY", "") // no flag, no env → no gateway + client, fc := testutil.NewFakeClient("default") + fc.AddSandbox("default", &types.Sandbox{Name: "agent-a", Status: types.SandboxStatus{Phase: types.SandboxReady}}) + + cmd := NewDeleteCmd(&mockGW{}, keepOpenFactory(client)) + cmd.SetArgs([]string{"--all"}) + _, err := captureStdout(t, cmd.Execute) + if err == nil { + t.Fatal("delete --all with no gateway should error, not report success") + } + if !contains(err.Error(), "no active openshell gateway") { + t.Errorf("unexpected error: %v", err) + } + + // Nothing was swept. + if names := sandboxNames(t, client); len(names) != 1 { + t.Errorf("no-gateway delete must not touch resources, got %v", names) + } +} + +// With no --gateway flag and no $OPENSHELL_GATEWAY, delete must fall back to the +// CLI's active-gateway marker (set by `openshell gateway select`) and actually +// sweep — not error, and not silently skip. This is the exact case the local +// integration teardowns hit: a gateway is selected but not pinned per-command. +func TestDeleteUsesActiveGateway(t *testing.T) { + t.Setenv("OPENSHELL_GATEWAY", "") // no flag, no env → must use the active marker + client, fc := testutil.NewFakeClient("default") + fc.AddSandbox("default", &types.Sandbox{Name: "agent-a", Status: types.SandboxStatus{Phase: types.SandboxReady}}) + + cmd := NewDeleteCmd(&mockGW{activeGateway: "openshell"}, keepOpenFactory(client)) + cmd.SetArgs([]string{"--sandboxes"}) + if _, err := captureStdout(t, cmd.Execute); err != nil { + t.Fatalf("delete --sandboxes with an active gateway: %v", err) + } + + if names := sandboxNames(t, client); len(names) != 0 { + t.Errorf("active-gateway fallback should sweep every sandbox, got %v", names) + } +} func TestDeleteProvidersSweep(t *testing.T) { client, fc := testutil.NewFakeClient("default") fc.AddProvider("default", &types.Provider{Name: "github", Type: "github"}) fc.AddProvider("default", &types.Provider{Name: "vertex", Type: "google-vertex-ai"}) - cmd := NewDeleteCmd("", "", keepOpenFactory(client)) + cmd := NewDeleteCmd(&mockGW{}, keepOpenFactory(client)) cmd.SetArgs([]string{"--providers", "--gateway", "prod"}) if _, err := captureStdout(t, cmd.Execute); err != nil { t.Fatalf("delete --providers: %v", err) diff --git a/cmd/deploy.go b/cmd/deploy.go deleted file mode 100644 index d4bef04..0000000 --- a/cmd/deploy.go +++ /dev/null @@ -1,314 +0,0 @@ -package cmd - -import ( - "context" - "fmt" - "os" - "os/exec" - "path/filepath" - "strings" - "time" - - "github.com/stackrox/harness-openshell/internal/gateway" - "github.com/stackrox/harness-openshell/internal/k8s" - "github.com/stackrox/harness-openshell/internal/status" - "github.com/spf13/cobra" -) - -func NewDeployCmd(harnessDir, cli string) *cobra.Command { - var kubeconfig string - - cmd := &cobra.Command{ - Use: "deploy [gateway]", - Short: "Deploy or verify the gateway", - Long: "Deploy a gateway by name (e.g., local-container, helm, openshift). Reads configuration from profiles/gateways/.yaml.", - Args: cobra.MaximumNArgs(1), - RunE: func(cmd *cobra.Command, args []string) error { - gatewayName, err := resolveGatewayName(args) - if err != nil { - return err - } - - gwCfg, err := resolveGatewayConfig(harnessDir, gatewayName) - if err != nil { - return fmt.Errorf("loading gateway config %q: %w", gatewayName, err) - } - - gw := gateway.New(cli) - - if gwCfg.IsLocal() { - return deployLocal(gw) - } - - kc := k8s.New(kubeconfig, k8s.DefaultNamespace()) - clusterRunner := k8s.New(kubeconfig, "") - return deployFromConfig(harnessDir, gwCfg, gw, kc, clusterRunner) - }, - } - - cmd.Flags().StringVar(&kubeconfig, "kubeconfig", "", "Path to kubeconfig (remote only)") - - return cmd -} - -func resolveGatewayName(args []string) (string, error) { - if len(args) > 0 { - return args[0], nil - } - return "", fmt.Errorf("specify a gateway: harness deploy ") -} - -// lookPath is exec.LookPath, overridable in tests to avoid a host -// dependency on podman. -var lookPath = exec.LookPath - -func deployLocal(gw gateway.Gateway) error { - cliPath := gw.CLIPath() - if cliPath == "" { - return fmt.Errorf("openshell CLI not found. Install it first:\n curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh") - } - - status.Header("Deploy") - if _, err := lookPath("podman"); err != nil { - status.Fail("Podman not found") - return fmt.Errorf("podman is required") - } - out, _ := exec.Command("podman", "--version").Output() - status.OKf("Podman: %s", strings.TrimSpace(string(out))) - gateways, err := gw.GatewayList() - if err != nil { - return fmt.Errorf("listing gateways: %w", err) - } - - var localGW string - for _, g := range gateways { - if strings.Contains(g.Endpoint, "127.0.0.1") { - localGW = g.Name - break - } - } - - if localGW == "" { - status.Fail("No local gateway found") - status.Detail("Install OpenShell (auto-registers the gateway):") - status.Sub("curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh") - return fmt.Errorf("no local gateway") - } - - if err := gw.GatewaySelect(localGW); err != nil { - return fmt.Errorf("selecting gateway %s: %w", localGW, err) - } - - // Retry InferenceGet a few times: the openshell daemon can briefly reload - // its config after a gateway add/select and take a few seconds to respond. - var inferErr error - for i := range 5 { - if inferErr = gw.InferenceGet(); inferErr == nil { - break - } - if i < 4 { - time.Sleep(3 * time.Second) - } - } - if inferErr == nil { - status.OKf("%s (active, reachable)", localGW) - } else { - status.Failf("%s (not responding)", localGW) - status.Detail("Start the gateway:") - status.Sub("macOS: brew services start openshell") - status.Sub("Linux: systemctl --user start openshell") - return fmt.Errorf("gateway not responding") - } - return nil -} - -func deployFromConfig(harnessDir string, gwCfg *gateway.GatewayConfig, gw gateway.Gateway, kc, clusterRunner k8s.Runner) (retErr error) { - defer func() { - if retErr != nil { - fmt.Fprintf(os.Stderr, "\nDeploy failed. Clean up with: harness delete --k8s\n") - } - }() - ctx := context.Background() - namespace := k8s.DefaultNamespace() - - tmpDir, err := os.MkdirTemp("", "harness-deploy-") - if err != nil { - return fmt.Errorf("creating temp dir: %w", err) - } - defer os.RemoveAll(tmpDir) - - chartVersion := os.Getenv("OPENSHELL_CHART_VERSION") - if chartVersion == "" { - chartVersion = gwCfg.Chart.Version - } - - status.Header("Deploy") - status.Infof("Chart: %s", chartVersion) - if kbcfg := os.Getenv("KUBECONFIG"); kbcfg != "" { - status.Infof("KUBECONFIG: %s", kbcfg) - } - - status.Step(1, "Namespace") - clusterRunner.RunKubectl(ctx, "create", "ns", namespace) - if _, err := clusterRunner.RunKubectl(ctx, "label", "ns", namespace, - "pod-security.kubernetes.io/enforce=privileged", - "pod-security.kubernetes.io/warn=privileged", - "--overwrite"); err != nil { - return fmt.Errorf("labeling namespace: %w", err) - } - - status.Step(2, "Sandbox CRD") - if _, err := clusterRunner.RunKubectl(ctx, "apply", "-f", gwCfg.Chart.CRD.URL); err != nil { - return fmt.Errorf("installing sandbox CRD: %w", err) - } - status.OK("Installed") - - if gwCfg.IsOCP() { - status.Step(3, "OpenShift SCCs") - for _, sa := range gwCfg.OCP.SCCPrivileged { - kc.RunOC(ctx, "adm", "policy", "add-scc-to-user", "privileged", "-z", sa, "-n", namespace) - } - for _, sa := range gwCfg.OCP.SCCAnyuid { - kc.RunOC(ctx, "adm", "policy", "add-scc-to-user", "anyuid", "-z", sa, "-n", namespace) - } - status.OK("Granted") - } - - // Addon manifests (RBAC, etc.) - for _, manifestPath := range gwCfg.ManifestFilePaths() { - if _, err := kc.RunKubectl(ctx, "apply", "-f", manifestPath); err != nil { - return fmt.Errorf("applying %s: %w", filepath.Base(manifestPath), err) - } - } - for _, manifest := range gwCfg.ManifestInline() { - if err := kc.ApplyYAML(ctx, manifest); err != nil { - return fmt.Errorf("applying inline manifest: %w", err) - } - } - - status.Step(4, "Helm install") - - // routeHost is needed before Helm (for OCP PKI cert SAN). - // gatewayURL is resolved after Helm for nodeport (service doesn't exist yet). - var routeHost string - if gwCfg.Gateway.Service == "route" { - appsDomain, err := clusterRunner.GetJSONPath(ctx, "ingresses.config.openshift.io/cluster", "{.spec.domain}") - if err != nil || appsDomain == "" { - return fmt.Errorf("could not determine OpenShift apps domain — is this an OpenShift cluster? (kubectl get ingresses.config.openshift.io cluster)") - } - routeHost = fmt.Sprintf("gateway-openshell.%s", appsDomain) - } - - helmArgs := []string{ - "upgrade", "--install", "openshell", gwCfg.Chart.OCI, - "--version", chartVersion, - } - if valuesPath, err := gwCfg.HelmValuesFile(tmpDir); err != nil { - return fmt.Errorf("helm values: %w", err) - } else if valuesPath != "" { - helmArgs = append(helmArgs, "--values", valuesPath) - } - if sandboxImage := os.Getenv("HARNESS_OS_IMAGE"); sandboxImage != "" { - helmArgs = append(helmArgs, "--set", "server.sandboxImage="+sandboxImage) - } - if routeHost != "" { - helmArgs = append(helmArgs, "--set", "pkiInitJob.serverDnsNames[0]="+routeHost) - } - if ps := os.Getenv("HARNESS_OS_PULL_SECRET"); ps != "" { - helmArgs = append(helmArgs, "--set", "imagePullSecrets[0].name="+ps) - } - if sps := os.Getenv("HARNESS_OS_SANDBOX_PULL_SECRET"); sps != "" { - helmArgs = append(helmArgs, "--set", "server.sandboxImagePullSecrets[0].name="+sps) - } - if err := kc.RunHelm(ctx, helmArgs...); err != nil { - return fmt.Errorf("helm install failed: %w", err) - } - - if _, err := kc.RunKubectl(ctx, "rollout", "status", "statefulset/openshell", "--timeout=300s"); err != nil { - return fmt.Errorf("gateway rollout failed: %w", err) - } - status.OK("Gateway ready") - - status.Step(5, "CLI gateway") - gatewayName := gwCfg.Gateway.Name - - var gatewayURL string - switch gwCfg.Gateway.Service { - case "route": - gatewayURL = fmt.Sprintf("https://%s:443", routeHost) - case "nodeport": - nodePort, err := kc.GetServiceNodePort(ctx, "openshell", 8080) - if err != nil { - return fmt.Errorf("getting NodePort: %w", err) - } - nodeIP, err := clusterRunner.GetNodeInternalIP(ctx) - if err != nil { - return fmt.Errorf("getting node IP: %w", err) - } - // Use HTTP — kind gateway runs with disableTls=true so the CLI - // registers plaintext, skipping mTLS and browser auth entirely. - gatewayURL = fmt.Sprintf("http://%s:%d", nodeIP, nodePort) - case "loadbalancer": - return fmt.Errorf("loadbalancer endpoint resolution not yet implemented") - } - - existing, err := gw.GatewayList() - if err != nil { - return fmt.Errorf("listing existing gateways: %w", err) - } - for _, g := range existing { - // Remove stale registration for same name or same route host (idempotent re-deploy). - if g.Name == gatewayName || (routeHost != "" && strings.Contains(g.Endpoint, routeHost)) { - gw.GatewayRemove(g.Name) - } - } - - if err := gw.GatewayAdd(gatewayURL, gatewayName, true, false); err != nil { - return fmt.Errorf("registering gateway %s: %w", gatewayName, err) - } - - // mTLS cert extraction — needed for remote clusters (OCP) where the - // gateway is exposed via TLS-passthrough Route. - if !gwCfg.IsLocal() && gwCfg.Secrets.MTLS != "" { - home, err := os.UserHomeDir() - if err != nil { - return fmt.Errorf("determining home directory: %w", err) - } - mtlsDir := filepath.Join(home, ".config", "openshell", "gateways", gatewayName, "mtls") - if err := os.MkdirAll(mtlsDir, 0o700); err != nil { - return fmt.Errorf("creating mtls directory: %w", err) - } - for _, field := range []string{"ca.crt", "tls.crt", "tls.key"} { - data, err := kc.GetSecretField(ctx, gwCfg.Secrets.MTLS, field) - if err != nil { - return fmt.Errorf("extracting %s from %s: %w", field, gwCfg.Secrets.MTLS, err) - } - if err := os.WriteFile(filepath.Join(mtlsDir, field), data, 0o600); err != nil { - return fmt.Errorf("writing %s: %w", field, err) - } - } - } - - if err := gw.GatewaySelect(gatewayName); err != nil { - return fmt.Errorf("selecting gateway %s: %w", gatewayName, err) - } - if !gwCfg.IsLocal() && gwCfg.Secrets.MTLS != "" { - status.OKf("%s registered (certs from cluster)", gatewayName) - } else { - status.OKf("%s registered", gatewayName) - } - - var gwReachable bool - for range 30 { - if gw.InferenceGet() == nil { - gwReachable = true - break - } - time.Sleep(2 * time.Second) - } - if !gwReachable { - return fmt.Errorf("gateway not reachable after 60s (try: openshell inference get)") - } - status.OK("Reachable") - return nil -} diff --git a/cmd/deploy_test.go b/cmd/deploy_test.go deleted file mode 100644 index 5a62279..0000000 --- a/cmd/deploy_test.go +++ /dev/null @@ -1,221 +0,0 @@ -package cmd - -import ( - "fmt" - "os" - "path/filepath" - "testing" - - "github.com/stackrox/harness-openshell/internal/gateway" - "github.com/stackrox/harness-openshell/internal/k8s" -) - -func setupDeployHarnessDir(t *testing.T) string { - t.Helper() - dir := t.TempDir() - os.WriteFile(filepath.Join(dir, "values-ocp.yaml"), []byte("image:\n pullPolicy: Always\n"), 0o644) - os.MkdirAll(filepath.Join(dir, "agents"), 0o755) - return dir -} - -func setupOCPGatewayConfig(t *testing.T, dir string) string { - t.Helper() - gwDir := filepath.Join(dir, "gateways", "openshift") - os.MkdirAll(filepath.Join(gwDir, "helm"), 0o755) - os.MkdirAll(filepath.Join(gwDir, "addons"), 0o755) - os.WriteFile(filepath.Join(gwDir, "gateway.yaml"), []byte(` -gateway: - type: remote - platform: ocp - service: route - name: test-ocp -chart: - oci: oci://ghcr.io/nvidia/openshell/helm-chart - version: "0.0.58" - crd: - url: https://example.com/crd.yaml -helm: - values: values.yaml -addons: - manifests: [addons/rbac.yaml, addons/route.yaml] -ocp: - scc-privileged: [sa1, sa2] - scc-anyuid: [sa1] -secrets: - names: [secret-a] - mtls: test-client-tls -`), 0o644) - os.WriteFile(filepath.Join(gwDir, "helm", "values.yaml"), []byte("image:\n pullPolicy: Always\n"), 0o644) - os.WriteFile(filepath.Join(gwDir, "addons", "rbac.yaml"), []byte("# rbac\n"), 0o644) - os.WriteFile(filepath.Join(gwDir, "addons", "route.yaml"), []byte("# route\n"), 0o644) - return gwDir -} - -func setupK8sGatewayConfig(t *testing.T, dir string) string { - t.Helper() - gwDir := filepath.Join(dir, "gateways", "helm") - os.MkdirAll(gwDir, 0o755) - os.WriteFile(filepath.Join(gwDir, "gateway.yaml"), []byte(` -gateway: - type: remote - platform: k8s - service: nodeport - name: test-kind - mode: direct -chart: - oci: oci://ghcr.io/nvidia/openshell/helm-chart - version: "0.0.58" - crd: - url: https://example.com/crd.yaml -`), 0o644) - return gwDir -} - -func TestDeployFromConfig_OCP_Success(t *testing.T) { - dir := setupDeployHarnessDir(t) - gwDir := setupOCPGatewayConfig(t, dir) - t.Setenv("OPENSHELL_CHART_VERSION", "0.0.58") - t.Setenv("OPENSHELL_NAMESPACE", "openshell") - t.Setenv("HOME", t.TempDir()) - - gwCfg, err := gateway.LoadConfig(gwDir) - if err != nil { - t.Fatal(err) - } - - nsRunner := k8s.NewMockRunner() - clusterRunner := k8s.NewMockRunner() - clusterRunner.Responses["get-jsonpath"] = "apps.example.com" - nsRunner.Responses["get-secret-field"] = "dGVzdA==" // base64 "test" - - gw := &mockGW{} - - err = deployFromConfig(dir, gwCfg, gw, nsRunner, clusterRunner) - if err != nil { - t.Fatalf("deployFromConfig: %v", err) - } - - // Verify namespace created - if !clusterRunner.HasCall("create ns openshell") { - t.Errorf("missing create ns, calls: %v", clusterRunner.Calls) - } - - // Verify CRD installed from config URL - if !clusterRunner.HasCall("apply -f https://example.com/crd.yaml") { - t.Errorf("missing CRD apply with config URL, calls: %v", clusterRunner.Calls) - } - - // Verify addon manifests applied (2: rbac + route) - if nsRunner.CallCount("apply -f") < 2 { - t.Errorf("expected >=2 apply -f calls for addon manifests, got %d: %v", nsRunner.CallCount("apply -f"), nsRunner.Calls) - } - - // Verify Helm install uses config chart OCI - if !nsRunner.HasCall("helm upgrade --install openshell oci://ghcr.io/nvidia/openshell/helm-chart") { - t.Errorf("missing helm install with config chart, calls: %v", nsRunner.Calls) - } - - // Verify rollout status - if !nsRunner.HasCall("rollout status statefulset/openshell") { - t.Errorf("missing rollout status, calls: %v", nsRunner.Calls) - } -} - -func TestDeployFromConfig_K8s_NoSCCs(t *testing.T) { - dir := setupDeployHarnessDir(t) - gwDir := setupK8sGatewayConfig(t, dir) - t.Setenv("OPENSHELL_CHART_VERSION", "0.0.58") - t.Setenv("OPENSHELL_NAMESPACE", "openshell") - t.Setenv("HOME", t.TempDir()) - - gwCfg, err := gateway.LoadConfig(gwDir) - if err != nil { - t.Fatal(err) - } - - nsRunner := k8s.NewMockRunner() - clusterRunner := k8s.NewMockRunner() - - gw := &mockGW{} - - // NodePort deploy should succeed — mock returns default node IP + port - err = deployFromConfig(dir, gwCfg, gw, nsRunner, clusterRunner) - if err != nil { - t.Fatalf("deployFromConfig: %v", err) - } - - // Verify NO OC/SCC calls were made (k8s, not OCP) - if nsRunner.HasCall("oc adm") { - t.Errorf("should not run oc commands on k8s platform, calls: %v", nsRunner.Calls) - } - - // Verify NO mTLS cert extraction (direct mode, no launcher) - if nsRunner.HasCall("get-secret-field") { - t.Errorf("should not extract mTLS certs for direct-mode k8s, calls: %v", nsRunner.Calls) - } -} - -func TestDeployFromConfig_HelmFailure(t *testing.T) { - dir := setupDeployHarnessDir(t) - gwDir := setupOCPGatewayConfig(t, dir) - t.Setenv("OPENSHELL_CHART_VERSION", "0.0.58") - t.Setenv("OPENSHELL_NAMESPACE", "openshell") - - gwCfg, err := gateway.LoadConfig(gwDir) - if err != nil { - t.Fatal(err) - } - - nsRunner := k8s.NewMockRunner() - clusterRunner := k8s.NewMockRunner() - clusterRunner.Responses["get-jsonpath"] = "apps.example.com" - nsRunner.Errors["helm upgrade"] = fmt.Errorf("chart not found") - - gw := &mockGW{} - - err = deployFromConfig(dir, gwCfg, gw, nsRunner, clusterRunner) - if err == nil { - t.Fatal("expected error from helm failure") - } - if nsRunner.HasCall("rollout status") { - t.Error("should not attempt rollout after helm failure") - } -} - -func TestDeployFromConfig_CRDFailure(t *testing.T) { - dir := setupDeployHarnessDir(t) - gwDir := setupOCPGatewayConfig(t, dir) - t.Setenv("OPENSHELL_CHART_VERSION", "0.0.58") - t.Setenv("OPENSHELL_NAMESPACE", "openshell") - - gwCfg, err := gateway.LoadConfig(gwDir) - if err != nil { - t.Fatal(err) - } - - nsRunner := k8s.NewMockRunner() - clusterRunner := k8s.NewMockRunner() - clusterRunner.Errors["apply -f"] = fmt.Errorf("network error") - - gw := &mockGW{} - - err = deployFromConfig(dir, gwCfg, gw, nsRunner, clusterRunner) - if err == nil { - t.Fatal("expected error from CRD install failure") - } - if nsRunner.HasCall("helm") { - t.Error("should not attempt helm after CRD failure") - } -} - -func TestDeployLocal_NoGateway(t *testing.T) { - gw := &mockGW{inferenceErr: fmt.Errorf("not reachable")} - gw.gatewayListResult = []gateway.GatewayInfo{ - {Name: "local", Endpoint: "https://127.0.0.1:17670"}, - } - - err := deployLocal(gw) - if err == nil { - t.Fatal("expected error for unreachable gateway") - } -} diff --git a/cmd/doctor.go b/cmd/doctor.go index 411e55b..94e339d 100644 --- a/cmd/doctor.go +++ b/cmd/doctor.go @@ -38,8 +38,8 @@ func NewDoctorCmd(harnessDir, cli string, newClient openshell.Factory) *cobra.Co Short: "Validate environment for configured sandbox", Long: `Check that prerequisites are met for running a sandbox. -Phase 1 (offline): checks openshell binary, target dependencies, and -provider credentials without requiring a running gateway. +Phase 1 (offline): checks the openshell binary and provider credentials +without requiring a running gateway. Phase 2 (online): if the gateway is reachable, checks provider registration.`, RunE: func(cmd *cobra.Command, args []string) error { @@ -55,7 +55,6 @@ Phase 2 (online): if the gateway is reachable, checks provider registration.`, checks := []CheckFunc{ checkOpenShell, - checkTargetDeps, checkProviderEnvVars, } @@ -128,70 +127,6 @@ func checkOpenShell(cfg *agent.AgentConfig, cli, _ string) []CheckResult { }} } -func checkTargetDeps(cfg *agent.AgentConfig, harnessDir, _ string) []CheckResult { - target := cfg.Gateway - if target == "" { - target = "local-container" - } - - gwCfg, _ := resolveGatewayConfig(harnessDir, target) - if gwCfg != nil { - if gwCfg.IsLocal() { - return checkLocalDeps() - } - return checkRemoteDeps() - } - - return checkLocalDeps() -} - -func checkLocalDeps() []CheckResult { - if _, err := exec.LookPath("podman"); err == nil { - if err := exec.Command("podman", "info").Run(); err == nil { - ver := "" - if out, e := exec.Command("podman", "version", "--format", "{{.Client.Version}}").Output(); e == nil { - ver = " " + strings.TrimSpace(string(out)) - } - return []CheckResult{{Group: "target", Name: "local-container", Status: "pass", Message: "podman" + ver + " running"}} - } - } - if _, err := exec.LookPath("docker"); err == nil { - if err := exec.Command("docker", "info").Run(); err == nil { - return []CheckResult{{Group: "target", Name: "local-container", Status: "pass", Message: "docker running"}} - } - } - return []CheckResult{{Group: "target", Name: "local-container", Status: "fail", Message: "no container runtime (podman or docker) responding"}} -} - -func checkRemoteDeps() []CheckResult { - var results []CheckResult - - kubectlFound := false - if _, err := exec.LookPath("kubectl"); err == nil { - kubectlFound = true - results = append(results, CheckResult{Group: "target", Name: "kubectl", Status: "pass", Message: "found"}) - } else if _, err := exec.LookPath("oc"); err == nil { - kubectlFound = true - results = append(results, CheckResult{Group: "target", Name: "oc", Status: "pass", Message: "found"}) - } - if !kubectlFound { - results = append(results, CheckResult{Group: "target", Name: "kubectl", Status: "fail", Message: "neither kubectl nor oc found on PATH"}) - } - - kubeconfig := os.Getenv("KUBECONFIG") - if kubeconfig == "" { - home, _ := os.UserHomeDir() - kubeconfig = filepath.Join(home, ".kube", "config") - } - if _, err := os.Stat(kubeconfig); err != nil { - results = append(results, CheckResult{Group: "target", Name: "kubeconfig", Status: "fail", Message: "kubeconfig not found at " + kubeconfig}) - } else { - results = append(results, CheckResult{Group: "target", Name: "kubeconfig", Status: "pass", Message: kubeconfig}) - } - - return results -} - type providerProfile struct { ID string `yaml:"id"` DisplayName string `yaml:"display_name"` @@ -453,10 +388,9 @@ func checkOnlineSDK(ctx context.Context, client openshell.Client, providers []st } func printDoctorTable(results []CheckResult) { - groups := []string{"openshell", "target", "provider", "gateway"} + groups := []string{"openshell", "provider", "gateway"} groupLabels := map[string]string{ "openshell": "OPENSHELL", - "target": "TARGET", "provider": "PROVIDER", "gateway": "GATEWAY", } diff --git a/cmd/doctor_test.go b/cmd/doctor_test.go index 9a17fdc..4aadf60 100644 --- a/cmd/doctor_test.go +++ b/cmd/doctor_test.go @@ -44,48 +44,6 @@ func TestCheckOpenShell_NotFound(t *testing.T) { } } -func TestCheckTargetDeps_Local(t *testing.T) { - cfg := testAgentConfig(t) - cfg.Gateway = "local-container" - results := checkTargetDeps(cfg, "", "") - if len(results) == 0 { - t.Fatal("expected at least 1 result") - } - if results[0].Group != "target" { - t.Errorf("Group = %q, want target", results[0].Group) - } -} - -func TestCheckTargetDeps_Remote(t *testing.T) { - cfg := testAgentConfig(t) - cfg.Gateway = "openshift" - results := checkTargetDeps(cfg, "", "") - if len(results) < 1 { - t.Fatal("expected at least 1 result for remote") - } - hasKubeconfig := false - for _, r := range results { - if r.Name == "kubeconfig" { - hasKubeconfig = true - } - } - if !hasKubeconfig { - t.Error("missing kubeconfig check for remote target") - } -} - -func TestCheckTargetDeps_EmptyGateway_DefaultsToLocal(t *testing.T) { - cfg := testAgentConfig(t) - cfg.Gateway = "" - results := checkTargetDeps(cfg, "", "") - if len(results) == 0 { - t.Fatal("expected at least 1 result") - } - if results[0].Name != "local-container" { - t.Errorf("Name = %q, want local-container (default)", results[0].Name) - } -} - func TestCheckProviderEnvVars_AllSet(t *testing.T) { dir := t.TempDir() writeProviderProfile(t, dir, "github", ` diff --git a/cmd/executor.go b/cmd/executor.go index e9c9c7b..1a0ec73 100644 --- a/cmd/executor.go +++ b/cmd/executor.go @@ -13,7 +13,6 @@ import ( "github.com/stackrox/harness-openshell/internal/agent" "github.com/stackrox/harness-openshell/internal/config" "github.com/stackrox/harness-openshell/internal/gateway" - "github.com/stackrox/harness-openshell/internal/k8s" "github.com/stackrox/harness-openshell/internal/openshell" "github.com/stackrox/harness-openshell/internal/payload" "github.com/stackrox/harness-openshell/internal/plan" @@ -34,8 +33,7 @@ var DefaultAgentConfig []byte type upLocalOpts struct { harnessDir string gw gateway.Gateway - gwCfg *gateway.GatewayConfig - ensureLocal bool + target openshell.Target agentCfg *agent.AgentConfig agentPath string sandboxName string @@ -71,19 +69,11 @@ func upLocal(opts upLocalOpts) error { status.Infof("Task: %s", agentCfg.Task) } - if opts.ensureLocal { - if err := deployLocal(gw); err != nil { - return fmt.Errorf("deploy failed: %w", err) - } - } else if gw.InferenceGet() != nil { - if opts.gwCfg == nil { - return fmt.Errorf("no active gateway -- use --gateway local or: harness deploy ocp") - } - kc := k8s.New("", k8s.DefaultNamespace()) - clusterRunner := k8s.New("", "") - if err := deployFromConfig(opts.harnessDir, opts.gwCfg, gw, kc, clusterRunner); err != nil { - return fmt.Errorf("deploy failed: %w", err) - } + // The harness no longer provisions gateways: apply runs against a gateway + // OpenShell already stood up and the user selected. Fail up front — before + // touching providers or creating a sandbox — if none is reachable. + if err := gw.InferenceGet(); err != nil { + return fmt.Errorf("no active gateway is reachable — provision one with the OpenShell installer or 'helm install openshell', then select it with 'openshell gateway select ': %w", err) } registered := ensureProviders(opts.harnessDir, gw, agentCfg, opts.harness) @@ -198,7 +188,7 @@ func upLocal(opts upLocalOpts) error { return run.RunSandbox(context.Background(), gw, run.SandboxRunRequest{ Name: sandboxName, - Gateway: gw.ActiveGateway(), + Gateway: opts.target.Gateway, Image: resolveSandboxImagePath(sandboxImage, opts.harnessDir), Providers: registered, Env: agentCfg.BuildEnvMap(), @@ -350,11 +340,10 @@ func reconcileGateway(opts upLocalOpts, agentCfg *agent.AgentConfig) { status.Warn("gateway reconcile skipped: no SDK client factory") return } - target, err := resolveApplyTarget(opts.gw) - if err != nil { - status.Warnf("gateway reconcile skipped: %v", err) - return - } + // Reconcile acts on the same target apply resolved once and threaded in — + // not a re-derivation — so providers/inference and the sandbox always land on + // the same gateway. + target := opts.target // Bound the whole reconcile: verify-by-default makes the inference write // contact the provider endpoint synchronously, so a stalled gateway or // endpoint would otherwise hang apply with no deadline. Every other failure diff --git a/cmd/executor_inference_test.go b/cmd/executor_inference_test.go index 3229664..73895f2 100644 --- a/cmd/executor_inference_test.go +++ b/cmd/executor_inference_test.go @@ -42,6 +42,7 @@ func TestUpLocal_InferenceReconcile_Create(t *testing.T) { err := upLocal(upLocalOpts{ harnessDir: dir, gw: gw, + target: openshell.Target{Gateway: "test-gw"}, agentPath: filepath.Join(dir, "agents", "default.yaml"), noTTY: true, newClient: keepOpenFactory(fakeClient), @@ -81,6 +82,7 @@ func TestUpLocal_InferenceReconcile_ModelChange(t *testing.T) { err := upLocal(upLocalOpts{ harnessDir: dir, gw: gw, + target: openshell.Target{Gateway: "test-gw"}, agentPath: filepath.Join(dir, "agents", "default.yaml"), noTTY: true, newClient: keepOpenFactory(fakeClient), @@ -109,6 +111,7 @@ func TestUpLocal_InferenceReconcile_ClientFailureDegrades(t *testing.T) { err := upLocal(upLocalOpts{ harnessDir: dir, gw: gw, + target: openshell.Target{Gateway: "test-gw"}, agentPath: filepath.Join(dir, "agents", "default.yaml"), noTTY: true, newClient: errFactory, @@ -131,6 +134,7 @@ func TestUpLocal_SetupOnly_SkipsSandbox(t *testing.T) { err := upLocal(upLocalOpts{ harnessDir: dir, gw: gw, + target: openshell.Target{Gateway: "test-gw"}, agentPath: filepath.Join(dir, "agents", "default.yaml"), noTTY: true, setupOnly: true, diff --git a/cmd/executor_provider_test.go b/cmd/executor_provider_test.go index e272e5c..cbaf1cb 100644 --- a/cmd/executor_provider_test.go +++ b/cmd/executor_provider_test.go @@ -7,6 +7,7 @@ import ( fake "github.com/NVIDIA/OpenShell/sdk/go/openshell/v1/fake" "github.com/NVIDIA/OpenShell/sdk/go/openshell/v1/types" + "github.com/stackrox/harness-openshell/internal/openshell" "github.com/stackrox/harness-openshell/internal/plan" "github.com/stackrox/harness-openshell/internal/testutil" ) @@ -37,6 +38,7 @@ func TestUpLocal_ProviderReconcile_AdoptsBootstrapped(t *testing.T) { err := upLocal(upLocalOpts{ harnessDir: dir, gw: gw, + target: openshell.Target{Gateway: "test-gw"}, agentPath: filepath.Join(dir, "agents", "default.yaml"), noTTY: true, setupOnly: true, @@ -67,6 +69,7 @@ func TestUpLocal_ProviderReconcile_OwnedNoop(t *testing.T) { err := upLocal(upLocalOpts{ harnessDir: dir, gw: gw, + target: openshell.Target{Gateway: "test-gw"}, agentPath: filepath.Join(dir, "agents", "default.yaml"), noTTY: true, newClient: keepOpenFactory(fakeClient), diff --git a/cmd/executor_test.go b/cmd/executor_test.go index ee95520..49a25a2 100644 --- a/cmd/executor_test.go +++ b/cmd/executor_test.go @@ -3,7 +3,6 @@ package cmd import ( "fmt" "os" - "os/exec" "path/filepath" "strings" "testing" @@ -166,34 +165,6 @@ func TestUpLocal_SandboxCreateOpts(t *testing.T) { } } -func TestUpLocal_EnsureLocal_DeploysGateway(t *testing.T) { - lookPath = func(string) (string, error) { return "/usr/bin/podman", nil } - t.Cleanup(func() { lookPath = exec.LookPath }) - - dir := setupTestAgent(t) - gw := &mockGW{ - providerList: []string{"github"}, - providers: map[string]bool{"github": true}, - gatewayListResult: []gateway.GatewayInfo{ - {Name: "local", Endpoint: "127.0.0.1:17670", Active: true}, - }, - } - - err := upLocal(upLocalOpts{ - harnessDir: dir, - gw: gw, - agentPath: filepath.Join(dir, "agents", "default.yaml"), - ensureLocal: true, - noTTY: true, - }) - if err != nil { - t.Fatalf("upLocal with ensureLocal=true: %v", err) - } - if gw.createCalls != 1 { - t.Fatalf("createCalls = %d, want 1", gw.createCalls) - } -} - func TestResolveHarness_EmbeddedFallback(t *testing.T) { dir := t.TempDir() DefaultAgentConfig = []byte(`name: embedded-default diff --git a/cmd/helpers_test.go b/cmd/helpers_test.go index f217064..4336548 100644 --- a/cmd/helpers_test.go +++ b/cmd/helpers_test.go @@ -9,14 +9,6 @@ import ( "github.com/stackrox/harness-openshell/internal/gateway" ) -func init() { - EmbeddedGatewayProfiles = map[string][]byte{ - "local-container": []byte("gateway:\n type: local\n"), - "helm": []byte("gateway:\n type: remote\n platform: k8s\n service: nodeport\n"), - "openshift": []byte("gateway:\n type: remote\n platform: ocp\n service: route\n"), - } -} - type mockGW struct { inferenceErr error providers map[string]bool diff --git a/cmd/init_cmd.go b/cmd/init_cmd.go index 63fa270..c91047f 100644 --- a/cmd/init_cmd.go +++ b/cmd/init_cmd.go @@ -27,7 +27,7 @@ var defaultProviders = []availableProvider{ {ID: "google-workspace", DisplayName: "Google Workspace", Category: "knowledge"}, } -func NewInitCmd(harnessDir string) *cobra.Command { +func NewInitCmd() *cobra.Command { var ( outputPath string force bool @@ -37,12 +37,12 @@ func NewInitCmd(harnessDir string) *cobra.Command { cmd := &cobra.Command{ Use: "init", Short: "Generate a harness.yaml config file", - Long: `Create a harness.yaml by selecting an entrypoint, providers, and -gateway target. The generated config is yours to version, share, and customize. + Long: `Create a harness.yaml by selecting an entrypoint and providers. +The generated config is yours to version, share, and customize. Use --non-interactive to write the embedded default config without prompts.`, RunE: func(cmd *cobra.Command, args []string) error { - return initRun(os.Stdin, os.Stdout, outputPath, force, nonInteractive, DefaultAgentConfig, harnessDir) + return initRun(os.Stdin, os.Stdout, outputPath, force, nonInteractive, DefaultAgentConfig) }, } @@ -53,7 +53,7 @@ Use --non-interactive to write the embedded default config without prompts.`, return cmd } -func initRun(in io.Reader, out io.Writer, outputPath string, force, nonInteractive bool, defaultCfg []byte, harnessDir string) error { +func initRun(in io.Reader, out io.Writer, outputPath string, force, nonInteractive bool, defaultCfg []byte) error { if _, err := os.Stat(outputPath); err == nil && !force { return fmt.Errorf("%s already exists (use --force to overwrite)", outputPath) } @@ -77,12 +77,6 @@ func initRun(in io.Reader, out io.Writer, outputPath string, force, nonInteracti return err } cfg.Providers = providers - - target, err := promptGateway(scanner, out, harnessDir) - if err != nil { - return err - } - cfg.Gateway = target } data, err := yaml.Marshal(cfg) @@ -141,26 +135,6 @@ func promptProviders(scanner *bufio.Scanner, out io.Writer) ([]agent.ProviderRef return buildProviderRefs(available, indices), nil } -func promptGateway(scanner *bufio.Scanner, out io.Writer, harnessDir string) (string, error) { - profiles := listGatewayProfiles(harnessDir) - defaultGW := "local-container" - choices := strings.Join(profiles, "/") - fmt.Fprintf(out, "Gateway target [%s] (default: %s): ", choices, defaultGW) - if !scanner.Scan() { - return defaultGW, nil - } - input := strings.TrimSpace(strings.ToLower(scanner.Text())) - if input == "" { - return defaultGW, nil - } - for _, p := range profiles { - if input == p { - return input, nil - } - } - return "", fmt.Errorf("unknown gateway target: %q (available: %s)", input, choices) -} - func discoverProviders() []availableProvider { if providers := discoverFromOpenShell(); len(providers) > 0 { return providers diff --git a/cmd/init_cmd_test.go b/cmd/init_cmd_test.go index c9f4b9e..56db2c7 100644 --- a/cmd/init_cmd_test.go +++ b/cmd/init_cmd_test.go @@ -25,7 +25,7 @@ func TestInitRun_NonInteractive(t *testing.T) { outPath := filepath.Join(dir, "harness.yaml") var buf bytes.Buffer - err := initRun(strings.NewReader(""), &buf, outPath, false, true, testDefaultConfig, "") + err := initRun(strings.NewReader(""), &buf, outPath, false, true, testDefaultConfig) if err != nil { t.Fatalf("initRun: %v", err) } @@ -53,7 +53,7 @@ func TestInitRun_OverwriteGuard(t *testing.T) { os.WriteFile(outPath, []byte("existing"), 0o644) var buf bytes.Buffer - err := initRun(strings.NewReader(""), &buf, outPath, false, true, testDefaultConfig, "") + err := initRun(strings.NewReader(""), &buf, outPath, false, true, testDefaultConfig) if err == nil { t.Fatal("expected error for existing file without --force") } @@ -68,7 +68,7 @@ func TestInitRun_OverwriteWithForce(t *testing.T) { os.WriteFile(outPath, []byte("existing"), 0o644) var buf bytes.Buffer - err := initRun(strings.NewReader(""), &buf, outPath, true, true, testDefaultConfig, "") + err := initRun(strings.NewReader(""), &buf, outPath, true, true, testDefaultConfig) if err != nil { t.Fatalf("initRun with --force: %v", err) } @@ -89,7 +89,7 @@ func TestInitRun_InteractiveDefaults(t *testing.T) { // Empty input = accept defaults for each prompt input := "\n\n\n" - err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig, "") + err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig) if err != nil { t.Fatalf("initRun: %v", err) } @@ -114,7 +114,7 @@ func TestInitRun_InteractiveOpenCode(t *testing.T) { var buf bytes.Buffer input := "opencode\n1\n\n" - err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig, "") + err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig) if err != nil { t.Fatalf("initRun: %v", err) } @@ -133,7 +133,7 @@ func TestInitRun_InteractiveProvidersSingle(t *testing.T) { var buf bytes.Buffer input := "claude\n1\n\n" - err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig, "") + err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig) if err != nil { t.Fatalf("initRun: %v", err) } @@ -152,7 +152,7 @@ func TestInitRun_InteractiveProvidersMultiple(t *testing.T) { var buf bytes.Buffer input := "claude\n1,3\n\n" - err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig, "") + err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig) if err != nil { t.Fatalf("initRun: %v", err) } @@ -171,7 +171,7 @@ func TestInitRun_InteractiveProvidersNone(t *testing.T) { var buf bytes.Buffer input := "claude\nnone\n\n" - err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig, "") + err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig) if err != nil { t.Fatalf("initRun: %v", err) } @@ -184,74 +184,12 @@ func TestInitRun_InteractiveProvidersNone(t *testing.T) { } } -func TestInitRun_InteractiveGatewayKind(t *testing.T) { - dir := t.TempDir() - outPath := filepath.Join(dir, "harness.yaml") - var buf bytes.Buffer - - input := "claude\n1\nhelm\n" - err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig, "") - if err != nil { - t.Fatalf("initRun: %v", err) - } - - data, _ := os.ReadFile(outPath) - var cfg agent.AgentConfig - yaml.Unmarshal(data, &cfg) - if cfg.Gateway != "helm" { - t.Errorf("Gateway = %q, want helm", cfg.Gateway) - } -} - -func TestInitRun_InteractiveGatewayOCP(t *testing.T) { - dir := t.TempDir() - outPath := filepath.Join(dir, "harness.yaml") - var buf bytes.Buffer - - input := "claude\n1\nopenshift\n" - err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig, "") - if err != nil { - t.Fatalf("initRun: %v", err) - } - - data, _ := os.ReadFile(outPath) - var cfg agent.AgentConfig - yaml.Unmarshal(data, &cfg) - if cfg.Gateway != "openshift" { - t.Errorf("Gateway = %q, want openshift", cfg.Gateway) - } -} - -func TestInitRun_InvalidGateway(t *testing.T) { - dir := t.TempDir() - outPath := filepath.Join(dir, "harness.yaml") - var buf bytes.Buffer - - input := "claude\n1\nbadtarget\n" - err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig, "") - if err == nil { - t.Fatal("expected error for invalid gateway target") - } -} - -func TestInitRun_RemoteIsInvalidGateway(t *testing.T) { - dir := t.TempDir() - outPath := filepath.Join(dir, "harness.yaml") - var buf bytes.Buffer - - input := "claude\n1\nremote\n" - err := initRun(strings.NewReader(input), &buf, outPath, false, false, testDefaultConfig, "") - if err == nil { - t.Fatal("expected error: 'remote' is not a valid gateway, use 'ocp'") - } -} - func TestInitRun_OutputContainsNextSteps(t *testing.T) { dir := t.TempDir() outPath := filepath.Join(dir, "harness.yaml") var buf bytes.Buffer - err := initRun(strings.NewReader(""), &buf, outPath, false, true, testDefaultConfig, "") + err := initRun(strings.NewReader(""), &buf, outPath, false, true, testDefaultConfig) if err != nil { t.Fatalf("initRun: %v", err) } @@ -325,7 +263,7 @@ func TestInitNoCredentialLeak(t *testing.T) { t.Setenv("ANTHROPIC_API_KEY", "sk-secret-key-12345") - err := initRun(strings.NewReader(""), &buf, outPath, false, true, testDefaultConfig, "") + err := initRun(strings.NewReader(""), &buf, outPath, false, true, testDefaultConfig) if err != nil { t.Fatalf("initRun: %v", err) } diff --git a/cmd/migrate_test.go b/cmd/migrate_test.go index eb3bce5..c04ae5c 100644 --- a/cmd/migrate_test.go +++ b/cmd/migrate_test.go @@ -47,8 +47,11 @@ entrypoint: claude` if migrated.Metadata.Name != "basic-test" { t.Errorf("name: got %q, want basic-test", migrated.Metadata.Name) } - if migrated.Spec.Target.Gateway != "rc-dev" { - t.Errorf("target.gateway: got %q, want rc-dev", migrated.Spec.Target.Gateway) + // The legacy gateway: field named a deploy profile, a concept the harness no + // longer owns; migration drops it rather than mismapping it to a registered + // gateway name, so spec.target.gateway is left empty for the user to set. + if migrated.Spec.Target.Gateway != "" { + t.Errorf("target.gateway: got %q, want empty (legacy gateway not carried)", migrated.Spec.Target.Gateway) } } diff --git a/cmd/resolve.go b/cmd/resolve.go index e2af7fa..c68a846 100644 --- a/cmd/resolve.go +++ b/cmd/resolve.go @@ -1,16 +1,12 @@ package cmd import ( - "errors" "fmt" "io/fs" "os" "path/filepath" - "sort" - "strings" "github.com/stackrox/harness-openshell/internal/agent" - "github.com/stackrox/harness-openshell/internal/gateway" ) @@ -95,15 +91,6 @@ func resolveBaseAgent(harnessDir string, overlay *agent.Harness) (*agent.Harness return overlay, nil } -func resolveGatewayConfigWithHarness(harnessDir, name string, h *agent.Harness) (*gateway.GatewayConfig, error) { - if h != nil { - if data, ok := h.Gateways[name]; ok { - return gateway.LoadConfigFromBytes(data) - } - } - return resolveGatewayConfig(harnessDir, name) -} - func versionedImage(name string) string { base := "quay.io/rcochran/openshell" if Version == "" || Version == "dev" { @@ -112,44 +99,6 @@ func versionedImage(name string) string { return base + ":" + name + "-" + Version } -// EmbeddedGatewayProfiles holds embedded gateway profile YAML, set from main.go. -var EmbeddedGatewayProfiles map[string][]byte - -func resolveGatewayConfig(harnessDir, name string) (*gateway.GatewayConfig, error) { - cfg, err := gateway.LoadProfile(harnessDir, name) - if err == nil { - return cfg, nil - } - if !errors.Is(err, fs.ErrNotExist) { - return nil, err - } - gwDir := filepath.Join(harnessDir, "gateways", name) - cfg, err = gateway.LoadConfig(gwDir) - if err == nil { - return cfg, nil - } - if !errors.Is(err, fs.ErrNotExist) { - return nil, err - } - if data, ok := EmbeddedGatewayProfiles[name]; ok { - return gateway.LoadConfigFromBytes(data) - } - return nil, fmt.Errorf("gateway profile %q not found", name) -} - -func resolveGatewayConfigFromFile(path string) (*gateway.GatewayConfig, error) { - data, err := os.ReadFile(path) - if err != nil { - return nil, fmt.Errorf("reading gateway profile %s: %w", path, err) - } - cfg, err := gateway.LoadConfigFromBytes(data) - if err != nil { - return nil, err - } - cfg.Dir = filepath.Dir(path) - return cfg, nil -} - func loadProviderProfiles(harnessDir string) map[string][]byte { profiles := make(map[string][]byte) dir := filepath.Join(harnessDir, "profiles", "providers") @@ -170,51 +119,3 @@ func loadProviderProfiles(harnessDir string) map[string][]byte { return profiles } -func loadGatewayProfile(harnessDir, name string) []byte { - path := filepath.Join(harnessDir, "profiles", "gateways", name+".yaml") - data, err := os.ReadFile(path) - if err == nil { - return data - } - if d, ok := EmbeddedGatewayProfiles[name]; ok { - return d - } - return nil -} - -func resolveFirstRemoteGateway(harnessDir string) *gateway.GatewayConfig { - for _, name := range listGatewayProfiles(harnessDir) { - cfg, err := resolveGatewayConfig(harnessDir, name) - if err == nil && !cfg.IsLocal() { - return cfg - } - } - return nil -} - -func listGatewayProfiles(harnessDir string) []string { - seen := make(map[string]bool) - for name := range EmbeddedGatewayProfiles { - seen[name] = true - } - dir := filepath.Join(harnessDir, "profiles", "gateways") - entries, err := os.ReadDir(dir) - if err == nil { - for _, e := range entries { - if e.IsDir() || !strings.HasSuffix(e.Name(), ".yaml") { - continue - } - if strings.ToLower(e.Name()) == "readme.md" { - continue - } - name := e.Name()[:len(e.Name())-5] - seen[name] = true - } - } - names := make([]string, 0, len(seen)) - for name := range seen { - names = append(names, name) - } - sort.Strings(names) - return names -} diff --git a/cmd/status_cmd.go b/cmd/status_cmd.go deleted file mode 100644 index ebc6071..0000000 --- a/cmd/status_cmd.go +++ /dev/null @@ -1,57 +0,0 @@ -package cmd - -import ( - "fmt" - - "github.com/stackrox/harness-openshell/internal/gateway" - "github.com/stackrox/harness-openshell/internal/status" - "github.com/spf13/cobra" -) - -func NewStatusCmd(harnessDir, cli string) *cobra.Command { - return &cobra.Command{ - Use: "status", - Short: "Show sandbox and gateway status", - RunE: func(cmd *cobra.Command, args []string) error { - gw := gateway.New(cli) - return runStatus(gw) - }, - } -} - -func runStatus(gw gateway.Gateway) error { - status.Header("Gateway") - active := gw.ActiveGateway() - if active != "" { - status.OKf("Active: %s", active) - ver := gw.CLIVersion() - if ver != "" { - status.Infof("CLI: %s", ver) - } - } else { - status.Info("No active gateway") - } - - fmt.Println() - status.Header("Sandboxes") - infos, err := gw.SandboxStatus() - if err != nil { - if active == "" { - status.Info("No active gateway, cannot list sandboxes") - return nil - } - return fmt.Errorf("listing sandboxes: %w", err) - } - if len(infos) == 0 { - status.Info("None running") - return nil - } - - headers := []string{"NAME", "PHASE"} - var rows [][]string - for _, info := range infos { - rows = append(rows, []string{info.Name, info.Phase}) - } - status.Table(headers, rows) - return nil -} diff --git a/cmd/status_cmd_test.go b/cmd/status_cmd_test.go deleted file mode 100644 index df2c3ec..0000000 --- a/cmd/status_cmd_test.go +++ /dev/null @@ -1,52 +0,0 @@ -package cmd - -import ( - "testing" - - "github.com/stackrox/harness-openshell/internal/gateway" -) - -type statusMockGW struct { - mockGW - statusResult []gateway.SandboxInfo - statusErr error - activeGW string - cliVer string -} - -func (m *statusMockGW) SandboxStatus() ([]gateway.SandboxInfo, error) { - return m.statusResult, m.statusErr -} -func (m *statusMockGW) ActiveGateway() string { return m.activeGW } -func (m *statusMockGW) CLIVersion() string { return m.cliVer } - -func TestRunStatus_DisplaysSandboxes(t *testing.T) { - gw := &statusMockGW{ - activeGW: "local", - cliVer: "openshell v0.0.58", - statusResult: []gateway.SandboxInfo{ - {Name: "agent", Phase: "Ready"}, - {Name: "test", Phase: "Stopped"}, - }, - } - if err := runStatus(gw); err != nil { - t.Fatalf("runStatus: %v", err) - } -} - -func TestRunStatus_NoSandboxes(t *testing.T) { - gw := &statusMockGW{ - activeGW: "local", - cliVer: "openshell v0.0.58", - } - if err := runStatus(gw); err != nil { - t.Fatalf("runStatus: %v", err) - } -} - -func TestRunStatus_NoGateway(t *testing.T) { - gw := &statusMockGW{} - if err := runStatus(gw); err != nil { - t.Fatalf("runStatus: %v", err) - } -} diff --git a/cmd/target.go b/cmd/target.go index 524c6f1..0db00c3 100644 --- a/cmd/target.go +++ b/cmd/target.go @@ -43,19 +43,26 @@ func openClient(ctx context.Context, newClient openshell.Factory, gatewayName, w // the CLI's currently-active gateway registration. // // Apply deliberately does NOT register the standard --gateway/--workspace target -// flags: apply's own --gateway flag names a deploy profile (e.g. "openshift"), -// not an openshell registration, so reusing it as the SDK target would connect -// to the wrong thing. Instead the registration name is read from the active -// gateway the CLI already selected. +// flags: the harness runs against whichever gateway OpenShell has selected, so +// the registration name is read from the active gateway the CLI already +// selected rather than pinned per-invocation. // -// An empty active gateway is an error, not a silent skip: without a registration -// name the SDK client cannot connect and inference reconcile would quietly -// no-op, hiding a misconfiguration. Workspace is left "" so sdkclient applies -// its "default" default (the single owner of that rule). +// $OPENSHELL_GATEWAY takes precedence over the CLI's persisted active-gateway +// marker, matching OpenShell's own request-targeting precedence: the env var +// overrides the target without moving the `*` in `openshell gateway list`, so +// reading only ActiveGateway() would wrongly reject an env-targeted apply. +// +// An empty target is an error, not a silent skip: the harness does not provision +// gateways (that is OpenShell's job), so without a selected registration there is +// nothing to run against. Workspace is left "" so sdkclient applies its "default" +// default (the single owner of that rule). func resolveApplyTarget(gw gateway.Gateway) (openshell.Target, error) { - name := gw.ActiveGateway() + name := os.Getenv(openshell.EnvGateway) + if name == "" { + name = gw.ActiveGateway() + } if name == "" { - return openshell.Target{}, fmt.Errorf("no active openshell gateway — deploy or select one first") + return openshell.Target{}, fmt.Errorf("no active openshell gateway — run 'openshell gateway select ' first (provision one with the OpenShell installer or 'helm install openshell')") } return openshell.Target{Gateway: name, Workspace: ""}, nil } diff --git a/cmd/target_test.go b/cmd/target_test.go index 5b324cb..c80f3f7 100644 --- a/cmd/target_test.go +++ b/cmd/target_test.go @@ -6,6 +6,7 @@ import ( ) func TestResolveApplyTarget_FromActiveGateway(t *testing.T) { + t.Setenv("OPENSHELL_GATEWAY", "") // isolate from the caller's environment gw := &mockGW{activeGateway: "prod-gw"} target, err := resolveApplyTarget(gw) @@ -20,7 +21,37 @@ func TestResolveApplyTarget_FromActiveGateway(t *testing.T) { } } +// $OPENSHELL_GATEWAY changes OpenShell's request target without moving the +// active-gateway marker, so apply must honor it over ActiveGateway(). +func TestResolveApplyTarget_EnvOverridesActiveGateway(t *testing.T) { + t.Setenv("OPENSHELL_GATEWAY", "env-gw") + gw := &mockGW{activeGateway: "active-gw"} + + target, err := resolveApplyTarget(gw) + if err != nil { + t.Fatalf("resolveApplyTarget: %v", err) + } + if target.Gateway != "env-gw" { + t.Errorf("Gateway = %q, want env-gw ($OPENSHELL_GATEWAY overrides active)", target.Gateway) + } +} + +// With no active-gateway marker, $OPENSHELL_GATEWAY alone is enough to target. +func TestResolveApplyTarget_EnvWithNoActiveGateway(t *testing.T) { + t.Setenv("OPENSHELL_GATEWAY", "env-gw") + gw := &mockGW{activeGateway: ""} + + target, err := resolveApplyTarget(gw) + if err != nil { + t.Fatalf("resolveApplyTarget: %v", err) + } + if target.Gateway != "env-gw" { + t.Errorf("Gateway = %q, want env-gw", target.Gateway) + } +} + func TestResolveApplyTarget_EmptyActiveGatewayErrors(t *testing.T) { + t.Setenv("OPENSHELL_GATEWAY", "") // no env override, no active gateway gw := &mockGW{activeGateway: ""} _, err := resolveApplyTarget(gw) diff --git a/cmd/teardown.go b/cmd/teardown.go deleted file mode 100644 index 7d77cce..0000000 --- a/cmd/teardown.go +++ /dev/null @@ -1,236 +0,0 @@ -package cmd - -import ( - "context" - "fmt" - "strings" - "time" - - "github.com/stackrox/harness-openshell/internal/gateway" - "github.com/stackrox/harness-openshell/internal/k8s" - "github.com/stackrox/harness-openshell/internal/status" - "github.com/spf13/cobra" -) - -func NewTeardownCmd(harnessDir, cli string) *cobra.Command { - var ( - sandboxes bool - providers bool - k8sFlag bool - ) - - cmd := &cobra.Command{ - Use: "teardown [--sandboxes] [--providers] [--k8s]", - Short: "Tear down sandboxes, providers, or k8s resources", - RunE: func(cmd *cobra.Command, args []string) error { - if !sandboxes && !providers && !k8sFlag { - return fmt.Errorf("specify at least one of --sandboxes, --providers, or --k8s") - } - - gw := gateway.New(cli) - activeGW := gw.ActiveGateway() - - if activeGW != "" { - status.Infof("Active gateway: %s", activeGW) - } else { - status.Info("Active gateway: none") - } - fmt.Println() - - if sandboxes { - teardownSandboxes(gw, activeGW) - } - if providers { - if err := teardownProviders(gw, activeGW); err != nil { - return err - } - } - if k8sFlag { - ns := k8s.DefaultNamespace() - gwCfg := resolveFirstRemoteGateway(harnessDir) - teardownK8s(gw, gwCfg, k8s.New("", ns), k8s.New("", "")) - } - - status.Done("Done.") - return nil - }, - } - - cmd.Flags().BoolVar(&sandboxes, "sandboxes", false, "Delete all sandboxes") - cmd.Flags().BoolVar(&providers, "providers", false, "Delete all providers") - cmd.Flags().BoolVar(&k8sFlag, "k8s", false, "Delete k8s resources") - - return cmd -} - -func teardownSandboxes(gw gateway.Gateway, activeGW string) { - status.Section("Sandboxes") - if activeGW == "" { - status.Info("No active gateway, skipping") - fmt.Println() - return - } - - names, err := gw.SandboxList() - if err != nil { - status.Fail(fmt.Sprintf("could not list sandboxes: %v", err)) - fmt.Println() - return - } - if len(names) == 0 { - status.Info("None running") - } else { - for _, name := range names { - status.Infof("Deleting %s", name) - if err := gw.SandboxDelete(name); err != nil { - status.Failf("failed to delete %s: %v", name, err) - } - } - } - fmt.Println() -} - -func teardownProviders(gw gateway.Gateway, activeGW string) error { - status.Section("Providers") - if activeGW == "" { - status.Info("No active gateway, skipping") - fmt.Println() - return nil - } - - remaining, err := gw.SandboxList() - if err != nil { - return fmt.Errorf("could not check for running sandboxes: %w", err) - } - if len(remaining) > 0 { - // Sandbox may be mid-deletion — wait briefly and retry - time.Sleep(2 * time.Second) - remaining, err = gw.SandboxList() - if err != nil { - return fmt.Errorf("rechecking sandboxes: %w", err) - } - if len(remaining) > 0 { - return fmt.Errorf("cannot delete providers with running sandboxes — run: harness teardown --sandboxes") - } - } - - names, err := gw.ProviderList() - if err != nil { - return fmt.Errorf("could not list providers: %w", err) - } - if len(names) == 0 { - status.Info("None registered") - } else { - for _, name := range names { - status.Infof("Deleting %s", name) - if err := gw.ProviderDelete(name); err != nil { - status.Failf("failed to delete %s: %v", name, err) - } - } - } - - fmt.Println() - return nil -} - -func teardownK8s(gw gateway.Gateway, gwCfg *gateway.GatewayConfig, kc, clusterRunner k8s.Runner) { - ctx := context.Background() - namespace := k8s.DefaultNamespace() - - // Resolve SCC SAs and secret names from config, or use defaults - sccSAs := []string{"openshell", "openshell-sandbox", "default"} - sccAnyuid := []string{"openshell"} - secrets := []string{"openshell-atlassian"} - if gwCfg != nil { - if len(gwCfg.OCP.SCCPrivileged) > 0 { - sccSAs = gwCfg.OCP.SCCPrivileged - } - if len(gwCfg.OCP.SCCAnyuid) > 0 { - sccAnyuid = gwCfg.OCP.SCCAnyuid - } - if len(gwCfg.Secrets.Names) > 0 { - secrets = gwCfg.Secrets.Names - } - } - - if !clusterRunner.NamespaceExists(ctx, namespace) { - status.Section("K8s") - status.Info("No openshell namespace found, skipping") - return - } - - // Helm release - status.Section("Helm release") - if err := kc.RunHelm(ctx, "uninstall", "openshell"); err == nil { - status.Info("Uninstalled") - } else { - status.Info("Not installed") - } - - // Sandbox CRD namespace - fmt.Println() - status.Section("Sandbox CRD") - if _, err := clusterRunner.RunKubectl(ctx, "delete", "ns", "agent-sandbox-system"); err == nil { - status.Info("Deleted agent-sandbox-system") - } else { - status.Info("Not found") - } - - // OpenShift SCCs - fmt.Println() - status.Section("OpenShift SCCs") - for _, sa := range sccSAs { - kc.RunOC(ctx, "adm", "policy", "remove-scc-from-user", "privileged", "-z", sa, "-n", namespace) - } - for _, sa := range sccAnyuid { - kc.RunOC(ctx, "adm", "policy", "remove-scc-from-user", "anyuid", "-z", sa, "-n", namespace) - } - // Clean up legacy cluster-admin binding (no longer created, but may exist from earlier deploys) - clusterRunner.RunKubectl(ctx, "delete", "clusterrolebinding", "agent-sandbox-admin") - status.Info("Cleared") - - // Secrets - fmt.Println() - status.Section("K8s secrets") - for _, secret := range secrets { - if _, err := kc.RunKubectl(ctx, "delete", "secret", secret); err == nil { - status.OKf("Deleted %s", secret) - } else { - status.Infof("%s: not found", secret) - } - } - - // Namespace - fmt.Println() - status.Section("Namespace") - if _, err := clusterRunner.RunKubectl(ctx, "delete", "ns", namespace); err == nil { - status.OKf("Deleted %s", namespace) - } else { - status.Infof("%s: not found", namespace) - } - - // Gateway config cleanup - fmt.Println() - status.Section("Gateway config") - gateways, err := gw.GatewayList() - if err != nil { - status.Failf("listing gateways: %v", err) - return - } - for _, g := range gateways { - if !strings.Contains(g.Endpoint, "127.0.0.1") { - if err := gw.GatewayRemove(g.Name); err == nil { - status.OKf("Removed gateway '%s'", g.Name) - } - } - } - // Select local gateway if available - for _, g := range gateways { - if strings.Contains(g.Endpoint, "127.0.0.1") { - gw.GatewaySelect(g.Name) - break - } - } - - fmt.Println() -} diff --git a/cmd/teardown_test.go b/cmd/teardown_test.go deleted file mode 100644 index 7a07a37..0000000 --- a/cmd/teardown_test.go +++ /dev/null @@ -1,63 +0,0 @@ -package cmd - -import ( - "fmt" - "testing" - - "github.com/stackrox/harness-openshell/internal/gateway" - "github.com/stackrox/harness-openshell/internal/k8s" -) - -func TestTeardownK8s_NamespaceNotFound(t *testing.T) { - kc := k8s.NewMockRunner() - clusterRunner := k8s.NewMockRunner() - clusterRunner.Errors["namespace-exists"] = fmt.Errorf("not found") - - gw := &mockGW{} - teardownK8s(gw, nil, kc, clusterRunner) - - if clusterRunner.HasCall("delete") { - t.Error("should not attempt deletes when namespace does not exist") - } -} - -func TestTeardownK8s_FullCleanup(t *testing.T) { - kc := k8s.NewMockRunner() - clusterRunner := k8s.NewMockRunner() - - deletedGateways := []string{} - gw := &mockGW{ - gatewayListResult: []gateway.GatewayInfo{ - {Name: "openshell-remote-ocp", Endpoint: "https://gw.example.com", Active: true}, - {Name: "openshell", Endpoint: "https://127.0.0.1:17670"}, - }, - onGatewayRemove: func(name string) { deletedGateways = append(deletedGateways, name) }, - } - - teardownK8s(gw, nil, kc, clusterRunner) - - // Should helm uninstall - if !kc.HasCall("helm uninstall") { - t.Errorf("expected helm uninstall, calls: %v", kc.Calls) - } - - // Should delete agent-sandbox-system namespace - if !clusterRunner.HasCall("delete ns agent-sandbox-system") { - t.Errorf("expected delete ns agent-sandbox-system, calls: %v", clusterRunner.Calls) - } - - // Should delete secrets - if kc.CallCount("delete secret") != 1 { - t.Errorf("expected 1 secret delete, got %d: %v", kc.CallCount("delete secret"), kc.Calls) - } - - // Should delete openshell namespace - if !clusterRunner.HasCall("delete ns openshell") { - t.Errorf("expected delete ns openshell, calls: %v", clusterRunner.Calls) - } - - // Should remove non-local gateway only - if len(deletedGateways) != 1 || deletedGateways[0] != "openshell-remote-ocp" { - t.Errorf("deleted gateways = %v, want [openshell-remote-ocp]", deletedGateways) - } -} diff --git a/internal/agent/agent.go b/internal/agent/agent.go index 68c2ee9..1370fb9 100644 --- a/internal/agent/agent.go +++ b/internal/agent/agent.go @@ -25,7 +25,6 @@ type PayloadEntry struct { type AgentConfig struct { Name string `yaml:"name"` BaseAgent string `yaml:"base_agent,omitempty"` - Gateway string `yaml:"gateway,omitempty"` Repo string `yaml:"repo,omitempty"` RepoRef string `yaml:"repo_ref,omitempty"` Providers []ProviderRef `yaml:"providers"` @@ -45,9 +44,6 @@ type AgentConfig struct { func (base *AgentConfig) MergeOver(overlay *AgentConfig) *AgentConfig { merged := *base merged.Name = overlay.Name - if overlay.Gateway != "" { - merged.Gateway = overlay.Gateway - } if overlay.Repo != "" { merged.Repo = overlay.Repo } diff --git a/internal/agent/agent_test.go b/internal/agent/agent_test.go index af8bd49..1689579 100644 --- a/internal/agent/agent_test.go +++ b/internal/agent/agent_test.go @@ -764,7 +764,6 @@ func TestMergeOver(t *testing.T) { base := &AgentConfig{ Name: "base", Entrypoint: "claude", - Gateway: "local-container", Providers: []ProviderRef{ {Profile: "github"}, {Profile: "google-vertex-ai"}, @@ -798,9 +797,6 @@ func TestMergeOver(t *testing.T) { if merged.Entrypoint != "claude" { t.Errorf("Entrypoint = %q, want claude (from base)", merged.Entrypoint) } - if merged.Gateway != "local-container" { - t.Errorf("Gateway = %q, want local-container (from base)", merged.Gateway) - } if merged.Repo != "https://github.com/stackrox/collector" { t.Errorf("Repo = %q, want stackrox/collector", merged.Repo) } diff --git a/internal/config/legacy/migrate.go b/internal/config/legacy/migrate.go index df393c5..4a772a9 100644 --- a/internal/config/legacy/migrate.go +++ b/internal/config/legacy/migrate.go @@ -47,17 +47,16 @@ func Migrate(legacy *agent.Harness) (*config.Harness, []Warning, error) { Kind: "Harness", Metadata: config.Metadata{Name: a.Name}, Spec: config.Spec{ - Target: config.Target{Gateway: a.Gateway}, + // spec.target.gateway names a registered OpenShell gateway and is + // left for the user to set: the legacy gateway: field named a deploy + // profile (local-container/helm/openshift), a concept the harness no + // longer owns (provisioning is OpenShell's job), so it has no + // v1alpha1 home and is not carried over. Source: config.Source{Repo: a.Repo, Ref: a.RepoRef}, Agent: config.Agent{Type: a.EffectiveEntrypoint()}, }, } - switch a.Gateway { - case "helm", "openshift", "local-container": - warn("gateway", fmt.Sprintf("deprecated gateway profile %q; set spec.target.gateway to a registered gateway name", a.Gateway)) - } - for _, p := range a.Providers { warn("providers[].profile", fmt.Sprintf("provider profile %q converted to a referenced provider", p.Profile)) h.Spec.Providers = append(h.Spec.Providers, config.Provider{Name: p.Profile, Management: "referenced"}) diff --git a/internal/config/legacy/migrate_test.go b/internal/config/legacy/migrate_test.go index 2cf315e..0d5cf57 100644 --- a/internal/config/legacy/migrate_test.go +++ b/internal/config/legacy/migrate_test.go @@ -84,8 +84,10 @@ func TestMigrateBasic(t *testing.T) { if migrated.Metadata.Name != "basic-test" { t.Errorf("name: got %q, want basic-test", migrated.Metadata.Name) } - if migrated.Spec.Target.Gateway != "rc-dev" { - t.Errorf("target.gateway: got %q, want rc-dev", migrated.Spec.Target.Gateway) + // The legacy gateway: field named a deploy profile, not a registered gateway, + // so migration leaves spec.target.gateway empty for the user to set. + if migrated.Spec.Target.Gateway != "" { + t.Errorf("target.gateway: got %q, want empty (legacy gateway not carried)", migrated.Spec.Target.Gateway) } if migrated.Spec.Source.Repo != "https://github.com/example/repo" { t.Errorf("source.repo: got %q, want https://github.com/example/repo", migrated.Spec.Source.Repo) @@ -213,7 +215,6 @@ func TestMigratePayloadRename(t *testing.T) { legacy := &agent.Harness{ Agent: &agent.AgentConfig{ Name: "rename-test", - Gateway: "rc-dev", Entrypoint: "claude", }, Payloads: []agent.PayloadEntry{ @@ -253,40 +254,6 @@ func TestMigratePayloadRename(t *testing.T) { } } -// TestMigrateDeprecatedGateway checks that deprecated gateway profiles emit warnings. -func TestMigrateDeprecatedGateway(t *testing.T) { - tests := []string{"helm", "openshift", "local-container"} - for _, profile := range tests { - legacy := &agent.Harness{ - Agent: &agent.AgentConfig{ - Name: "deprecated-test", - Gateway: profile, - Entrypoint: "claude", - }, - } - - _, warnings, err := Migrate(legacy) - if err != nil { - t.Fatalf("migrate %q: %v", profile, err) - } - - if len(warnings) == 0 { - t.Errorf("gateway %q: expected deprecation warning, got none", profile) - } - - found := false - for _, w := range warnings { - if w.Field == "gateway" { - found = true - break - } - } - if !found { - t.Errorf("gateway %q: no warning with field=gateway", profile) - } - } -} - // TestMigrateInlineDocsWarn checks that inline kind:gateway and kind:provider // documents are flagged rather than silently dropped. func TestMigrateInlineDocsWarn(t *testing.T) { @@ -366,9 +333,10 @@ providers: t.Fatalf("MigrateBytes: %v", err) } - // Should have 2 warnings: 1 for deprecated gateway, 1 for provider profile - if len(warnings) < 2 { - t.Errorf("warnings count: got %d, want at least 2", len(warnings)) + // The legacy gateway: field is dropped without a warning (it named a removed + // deploy-profile concept); the provider profile still warns on conversion. + if len(warnings) < 1 { + t.Errorf("warnings count: got %d, want at least 1", len(warnings)) } // Validate output still parses @@ -397,8 +365,7 @@ func TestMigrateNoAgent(t *testing.T) { func TestMigrateDefaultEntrypoint(t *testing.T) { legacy := &agent.Harness{ Agent: &agent.AgentConfig{ - Name: "default-entrypoint-test", - Gateway: "rc-dev", + Name: "default-entrypoint-test", // Entrypoint is empty }, } diff --git a/internal/config/legacy/testdata/golden/basic.v1alpha1.yaml b/internal/config/legacy/testdata/golden/basic.v1alpha1.yaml index 38052f8..c6c5523 100644 --- a/internal/config/legacy/testdata/golden/basic.v1alpha1.yaml +++ b/internal/config/legacy/testdata/golden/basic.v1alpha1.yaml @@ -3,8 +3,7 @@ kind: Harness metadata: name: basic-test spec: - target: - gateway: rc-dev + target: {} agent: type: claude source: diff --git a/internal/config/legacy/testdata/golden/deprecated-gateway.v1alpha1.yaml b/internal/config/legacy/testdata/golden/deprecated-gateway.v1alpha1.yaml deleted file mode 100644 index f53b540..0000000 --- a/internal/config/legacy/testdata/golden/deprecated-gateway.v1alpha1.yaml +++ /dev/null @@ -1,12 +0,0 @@ -apiVersion: harness.openshell.dev/v1alpha1 -kind: Harness -metadata: - name: deprecated-gateway-test -spec: - target: - gateway: helm - agent: - type: claude - source: - repo: https://github.com/example/repo - ref: main diff --git a/internal/config/legacy/testdata/golden/sandbox-fields.v1alpha1.yaml b/internal/config/legacy/testdata/golden/sandbox-fields.v1alpha1.yaml index 4f458f5..e84bbf6 100644 --- a/internal/config/legacy/testdata/golden/sandbox-fields.v1alpha1.yaml +++ b/internal/config/legacy/testdata/golden/sandbox-fields.v1alpha1.yaml @@ -3,8 +3,7 @@ kind: Harness metadata: name: sandbox-fields-test spec: - target: - gateway: rc-dev + target: {} sandbox: image: quay.io/example/sandbox:latest policy: diff --git a/internal/config/legacy/testdata/golden/with-payloads.v1alpha1.yaml b/internal/config/legacy/testdata/golden/with-payloads.v1alpha1.yaml index 87d93a5..14dd9b1 100644 --- a/internal/config/legacy/testdata/golden/with-payloads.v1alpha1.yaml +++ b/internal/config/legacy/testdata/golden/with-payloads.v1alpha1.yaml @@ -3,8 +3,7 @@ kind: Harness metadata: name: with-payloads-test spec: - target: - gateway: rc-dev + target: {} agent: type: claude source: diff --git a/internal/config/legacy/testdata/golden/with-providers.v1alpha1.yaml b/internal/config/legacy/testdata/golden/with-providers.v1alpha1.yaml index 7cb2871..1327cf9 100644 --- a/internal/config/legacy/testdata/golden/with-providers.v1alpha1.yaml +++ b/internal/config/legacy/testdata/golden/with-providers.v1alpha1.yaml @@ -3,8 +3,7 @@ kind: Harness metadata: name: with-providers-test spec: - target: - gateway: rc-dev + target: {} providers: - name: github-fact management: referenced diff --git a/internal/config/legacy/testdata/legacy/deprecated-gateway.yaml b/internal/config/legacy/testdata/legacy/deprecated-gateway.yaml deleted file mode 100644 index 9e361d0..0000000 --- a/internal/config/legacy/testdata/legacy/deprecated-gateway.yaml +++ /dev/null @@ -1,5 +0,0 @@ -name: deprecated-gateway-test -gateway: helm -repo: https://github.com/example/repo -repo_ref: main -entrypoint: claude diff --git a/internal/gateway/config.go b/internal/gateway/config.go deleted file mode 100644 index 504427e..0000000 --- a/internal/gateway/config.go +++ /dev/null @@ -1,214 +0,0 @@ -package gateway - -import ( - "fmt" - "os" - "path/filepath" - - "gopkg.in/yaml.v3" -) - -type GatewayConfig struct { - Gateway GatewaySection `yaml:"gateway"` - Providers ProvidersSection `yaml:"providers"` - Chart ChartSection `yaml:"chart"` - Helm HelmSection `yaml:"helm"` - Addons AddonsSection `yaml:"addons"` - OCP OCPSection `yaml:"ocp"` - Secrets SecretsSection `yaml:"secrets"` - - Dir string `yaml:"-"` -} - -type GatewaySection struct { - Type string `yaml:"type"` - Platform string `yaml:"platform"` - Service string `yaml:"service"` - Name string `yaml:"name"` - Mode string `yaml:"mode"` -} - -type ProvidersSection struct { - Enabled []string `yaml:"enabled"` - Custom []string `yaml:"custom"` -} - -type ChartSection struct { - OCI string `yaml:"oci"` - Version string `yaml:"version"` - CRD CRDConfig `yaml:"crd"` -} - -type CRDConfig struct { - URL string `yaml:"url"` -} - -type HelmSection struct { - ValuesPath string `yaml:"-"` - ValuesInline map[string]any `yaml:"-"` -} - -func (h *HelmSection) UnmarshalYAML(value *yaml.Node) error { - var raw struct { - Values yaml.Node `yaml:"values"` - } - if err := value.Decode(&raw); err != nil { - return err - } - if raw.Values.Kind == 0 { - return nil - } - switch raw.Values.Kind { - case yaml.ScalarNode: - h.ValuesPath = raw.Values.Value - case yaml.MappingNode: - var m map[string]any - if err := raw.Values.Decode(&m); err != nil { - return fmt.Errorf("decoding inline helm values: %w", err) - } - h.ValuesInline = m - } - return nil -} - -type ManifestRef struct { - Path string `yaml:"-"` - Inline map[string]any `yaml:"-"` -} - -type AddonsSection struct { - Manifests []ManifestRef -} - -func (a *AddonsSection) UnmarshalYAML(value *yaml.Node) error { - var raw struct { - Manifests []yaml.Node `yaml:"manifests"` - } - if err := value.Decode(&raw); err != nil { - return err - } - for _, node := range raw.Manifests { - var ref ManifestRef - switch node.Kind { - case yaml.ScalarNode: - ref.Path = node.Value - case yaml.MappingNode: - var m map[string]any - if err := node.Decode(&m); err != nil { - return fmt.Errorf("decoding inline manifest: %w", err) - } - ref.Inline = m - } - a.Manifests = append(a.Manifests, ref) - } - return nil -} - -type OCPSection struct { - SCCPrivileged []string `yaml:"scc-privileged"` - SCCAnyuid []string `yaml:"scc-anyuid"` -} - -type SecretsSection struct { - Names []string `yaml:"names"` - MTLS string `yaml:"mtls"` -} - -func LoadConfig(dir string) (*GatewayConfig, error) { - data, err := os.ReadFile(filepath.Join(dir, "gateway.yaml")) - if err != nil { - return nil, fmt.Errorf("reading gateway config: %w", err) - } - cfg, err := LoadConfigFromBytes(data) - if err != nil { - return nil, err - } - cfg.Dir = dir - return cfg, nil -} - -func LoadConfigFromBytes(data []byte) (*GatewayConfig, error) { - var cfg GatewayConfig - if err := yaml.Unmarshal(data, &cfg); err != nil { - return nil, fmt.Errorf("parsing gateway config: %w", err) - } - cfg.applyDefaults() - cfg.applyEnvOverrides() - return &cfg, nil -} - -func LoadProfile(harnessDir, name string) (*GatewayConfig, error) { - path := filepath.Join(harnessDir, "profiles", "gateways", name+".yaml") - data, err := os.ReadFile(path) - if err != nil { - return nil, err - } - return LoadConfigFromBytes(data) -} - -func (c *GatewayConfig) applyDefaults() { - if c.Chart.OCI == "" { - c.Chart.OCI = "oci://ghcr.io/nvidia/openshell/helm-chart" - } - if c.Chart.CRD.URL == "" { - // Pin to the agent-sandbox release OpenShell itself pins (see the - // upstream e2e/with-kube-gateway.sh + helm-k3s-local.sh, which use - // AGENT_SANDBOX_VERSION=v0.5.0). v0.5.0's manifest.yaml carries both the - // v1beta1 and v1alpha1 Sandbox APIs plus the controller. Do NOT track - // releases/latest: latest moved to v0.5.6, which renamed the manifest.yaml - // asset to sandbox.yaml and would 404 here. - c.Chart.CRD.URL = "https://github.com/kubernetes-sigs/agent-sandbox/releases/download/v0.5.0/manifest.yaml" - } -} - -func (c *GatewayConfig) applyEnvOverrides() { - if v := os.Getenv("HARNESS_OS_GATEWAY"); v != "" { - c.Gateway.Name = v - } -} - -func (c *GatewayConfig) IsLocal() bool { - return c.Gateway.Type == "local" -} - -func (c *GatewayConfig) IsOCP() bool { - return c.Gateway.Platform == "ocp" -} - -func (c *GatewayConfig) HelmValuesFile(tmpDir string) (string, error) { - if c.Helm.ValuesInline != nil { - data, err := yaml.Marshal(c.Helm.ValuesInline) - if err != nil { - return "", fmt.Errorf("marshaling inline helm values: %w", err) - } - path := filepath.Join(tmpDir, "values.yaml") - if err := os.WriteFile(path, data, 0o644); err != nil { - return "", err - } - return path, nil - } - if c.Helm.ValuesPath == "" { - return "", nil - } - return filepath.Join(c.Dir, "helm", c.Helm.ValuesPath), nil -} - -func (c *GatewayConfig) ManifestFilePaths() []string { - var paths []string - for _, m := range c.Addons.Manifests { - if m.Path != "" { - paths = append(paths, filepath.Join(c.Dir, m.Path)) - } - } - return paths -} - -func (c *GatewayConfig) ManifestInline() []map[string]any { - var manifests []map[string]any - for _, m := range c.Addons.Manifests { - if m.Inline != nil { - manifests = append(manifests, m.Inline) - } - } - return manifests -} diff --git a/internal/gateway/config_test.go b/internal/gateway/config_test.go deleted file mode 100644 index b62e752..0000000 --- a/internal/gateway/config_test.go +++ /dev/null @@ -1,425 +0,0 @@ -package gateway - -import ( - "os" - "path/filepath" - "strings" - "testing" -) - -func writeGatewayYAML(t *testing.T, dir, content string) { - t.Helper() - if err := os.MkdirAll(dir, 0o755); err != nil { - t.Fatal(err) - } - if err := os.WriteFile(filepath.Join(dir, "gateway.yaml"), []byte(content), 0o644); err != nil { - t.Fatal(err) - } -} - -func TestLoadConfig_FullOCP(t *testing.T) { - dir := t.TempDir() - writeGatewayYAML(t, dir, ` -gateway: - type: remote - platform: ocp - service: route - name: my-ocp - -providers: - enabled: [github, google-vertex-ai] - custom: [google-workspace] - -chart: - oci: oci://example.com/chart - version: "1.2.3" - crd: - url: https://example.com/crd.yaml - -helm: - values: values.yaml - -addons: - manifests: [addons/route.yaml] - -ocp: - scc-privileged: [sa1, sa2] - scc-anyuid: [sa1] - -secrets: - names: [secret-a, secret-b] - mtls: my-mtls-secret -`) - - cfg, err := LoadConfig(dir) - if err != nil { - t.Fatal(err) - } - - if cfg.Gateway.Type != "remote" { - t.Errorf("type = %q, want remote", cfg.Gateway.Type) - } - if cfg.Gateway.Platform != "ocp" { - t.Errorf("platform = %q, want ocp", cfg.Gateway.Platform) - } - if cfg.Gateway.Service != "route" { - t.Errorf("service = %q, want route", cfg.Gateway.Service) - } - if cfg.Gateway.Name != "my-ocp" { - t.Errorf("name = %q, want my-ocp", cfg.Gateway.Name) - } - if len(cfg.Providers.Enabled) != 2 { - t.Errorf("providers.enabled = %v, want 2 entries", cfg.Providers.Enabled) - } - if len(cfg.Providers.Custom) != 1 || cfg.Providers.Custom[0] != "google-workspace" { - t.Errorf("providers.custom = %v, want [google-workspace]", cfg.Providers.Custom) - } - if cfg.Chart.OCI != "oci://example.com/chart" { - t.Errorf("chart.oci = %q, want oci://example.com/chart", cfg.Chart.OCI) - } - if cfg.Chart.Version != "1.2.3" { - t.Errorf("chart.version = %q, want 1.2.3", cfg.Chart.Version) - } - if cfg.Chart.CRD.URL != "https://example.com/crd.yaml" { - t.Errorf("chart.crd.url = %q", cfg.Chart.CRD.URL) - } - if len(cfg.OCP.SCCPrivileged) != 2 { - t.Errorf("ocp.scc-privileged = %v, want 2 entries", cfg.OCP.SCCPrivileged) - } - if len(cfg.OCP.SCCAnyuid) != 1 { - t.Errorf("ocp.scc-anyuid = %v, want 1 entry", cfg.OCP.SCCAnyuid) - } - if cfg.Secrets.MTLS != "my-mtls-secret" { - t.Errorf("secrets.mtls = %q", cfg.Secrets.MTLS) - } - if cfg.Dir != dir { - t.Errorf("Dir = %q, want %q", cfg.Dir, dir) - } -} - -func TestLoadConfig_MinimalLocal(t *testing.T) { - dir := t.TempDir() - writeGatewayYAML(t, dir, ` -gateway: - type: local -`) - - cfg, err := LoadConfig(dir) - if err != nil { - t.Fatal(err) - } - - if !cfg.IsLocal() { - t.Error("IsLocal() = false, want true") - } - - if cfg.Chart.OCI != "oci://ghcr.io/nvidia/openshell/helm-chart" { - t.Errorf("default chart.oci = %q", cfg.Chart.OCI) - } - if cfg.Secrets.MTLS != "" { - t.Errorf("default secrets.mtls = %q, want empty", cfg.Secrets.MTLS) - } -} - -func TestLoadConfig_MinimalRemote(t *testing.T) { - dir := t.TempDir() - writeGatewayYAML(t, dir, ` -gateway: - type: remote - platform: k8s -`) - - cfg, err := LoadConfig(dir) - if err != nil { - t.Fatal(err) - } - - if cfg.IsLocal() { - t.Error("IsLocal() = true for remote") - } - if cfg.IsOCP() { - t.Error("IsOCP() = true for k8s") - } -} - -func TestLoadConfig_Missing(t *testing.T) { - _, err := LoadConfig(t.TempDir()) - if err == nil { - t.Error("expected error for missing gateway.yaml") - } -} - -func TestLoadConfig_InvalidYAML(t *testing.T) { - dir := t.TempDir() - writeGatewayYAML(t, dir, `gateway: [broken yaml`) - - _, err := LoadConfig(dir) - if err == nil { - t.Error("expected error for invalid YAML") - } -} - -func TestEnvOverrides(t *testing.T) { - dir := t.TempDir() - writeGatewayYAML(t, dir, ` -gateway: - type: remote - name: original-name -`) - - t.Setenv("HARNESS_OS_GATEWAY", "env-gw-name") - - cfg, err := LoadConfig(dir) - if err != nil { - t.Fatal(err) - } - - if cfg.Gateway.Name != "env-gw-name" { - t.Errorf("HARNESS_OS_GATEWAY override: got %q", cfg.Gateway.Name) - } -} - -func TestEnvOverrides_NotSet(t *testing.T) { - dir := t.TempDir() - writeGatewayYAML(t, dir, ` -gateway: - type: remote - name: original-name -`) - - t.Setenv("HARNESS_OS_GATEWAY", "") - - cfg, err := LoadConfig(dir) - if err != nil { - t.Fatal(err) - } - - if cfg.Gateway.Name != "original-name" { - t.Errorf("expected original value, got %q", cfg.Gateway.Name) - } -} - -func TestHelmValuesFile_Path(t *testing.T) { - dir := t.TempDir() - writeGatewayYAML(t, dir, ` -gateway: - type: remote -helm: - values: values.yaml -`) - - cfg, err := LoadConfig(dir) - if err != nil { - t.Fatal(err) - } - - tmpDir := t.TempDir() - want := filepath.Join(dir, "helm", "values.yaml") - got, err := cfg.HelmValuesFile(tmpDir) - if err != nil { - t.Fatal(err) - } - if got != want { - t.Errorf("HelmValuesFile() = %q, want %q", got, want) - } -} - -func TestHelmValuesFile_Inline(t *testing.T) { - dir := t.TempDir() - writeGatewayYAML(t, dir, ` -gateway: - type: remote -helm: - values: - service: - type: NodePort - server: - disableTls: true -`) - - cfg, err := LoadConfig(dir) - if err != nil { - t.Fatal(err) - } - - tmpDir := t.TempDir() - got, err := cfg.HelmValuesFile(tmpDir) - if err != nil { - t.Fatal(err) - } - if got == "" { - t.Fatal("HelmValuesFile() returned empty path for inline values") - } - data, err := os.ReadFile(got) - if err != nil { - t.Fatal(err) - } - if !strings.Contains(string(data), "NodePort") { - t.Errorf("inline values file missing NodePort: %s", data) - } -} - -func TestHelmValuesFile_Empty(t *testing.T) { - dir := t.TempDir() - writeGatewayYAML(t, dir, ` -gateway: - type: remote -`) - - cfg, err := LoadConfig(dir) - if err != nil { - t.Fatal(err) - } - - tmpDir := t.TempDir() - got, err := cfg.HelmValuesFile(tmpDir) - if err != nil { - t.Fatal(err) - } - if got != "" { - t.Errorf("HelmValuesFile() = %q, want empty", got) - } -} - -func TestManifestFilePaths(t *testing.T) { - dir := t.TempDir() - writeGatewayYAML(t, dir, ` -gateway: - type: remote -addons: - manifests: - - addons/rbac.yaml - - addons/route.yaml -`) - - cfg, err := LoadConfig(dir) - if err != nil { - t.Fatal(err) - } - - paths := cfg.ManifestFilePaths() - if len(paths) != 2 { - t.Fatalf("ManifestFilePaths() returned %d paths, want 2", len(paths)) - } - if paths[0] != filepath.Join(dir, "addons", "rbac.yaml") { - t.Errorf("paths[0] = %q", paths[0]) - } - if paths[1] != filepath.Join(dir, "addons", "route.yaml") { - t.Errorf("paths[1] = %q", paths[1]) - } -} - -func TestManifestInline(t *testing.T) { - dir := t.TempDir() - writeGatewayYAML(t, dir, ` -gateway: - type: remote -addons: - manifests: - - apiVersion: route.openshift.io/v1 - kind: Route - metadata: - name: gateway -`) - - cfg, err := LoadConfig(dir) - if err != nil { - t.Fatal(err) - } - - inline := cfg.ManifestInline() - if len(inline) != 1 { - t.Fatalf("ManifestInline() returned %d, want 1", len(inline)) - } - if inline[0]["kind"] != "Route" { - t.Errorf("expected kind=Route, got %v", inline[0]["kind"]) - } -} - -func TestManifests_Empty(t *testing.T) { - dir := t.TempDir() - writeGatewayYAML(t, dir, ` -gateway: - type: remote -`) - - cfg, err := LoadConfig(dir) - if err != nil { - t.Fatal(err) - } - - if len(cfg.ManifestFilePaths()) != 0 { - t.Errorf("ManifestFilePaths() should be empty") - } - if len(cfg.ManifestInline()) != 0 { - t.Errorf("ManifestInline() should be empty") - } -} - -func TestLoadConfigFromBytes(t *testing.T) { - data := []byte(` -gateway: - type: remote - platform: ocp -chart: - version: "0.0.59" -`) - cfg, err := LoadConfigFromBytes(data) - if err != nil { - t.Fatal(err) - } - if cfg.Gateway.Platform != "ocp" { - t.Errorf("platform = %q, want ocp", cfg.Gateway.Platform) - } - if cfg.Chart.OCI == "" { - t.Error("defaults not applied") - } -} - -func TestPredicates(t *testing.T) { - tests := []struct { - name string - yaml string - isLocal bool - isOCP bool - }{ - { - name: "local", - yaml: "gateway:\n type: local", - isLocal: true, - isOCP: false, - }, - { - name: "remote ocp launcher", - yaml: "gateway:\n type: remote\n platform: ocp", - isLocal: false, - isOCP: true, - }, - { - name: "remote k8s direct", - yaml: "gateway:\n type: remote\n platform: k8s", - isLocal: false, - isOCP: false, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - dir := t.TempDir() - writeGatewayYAML(t, dir, tt.yaml) - - cfg, err := LoadConfig(dir) - if err != nil { - t.Fatal(err) - } - - if cfg.IsLocal() != tt.isLocal { - t.Errorf("IsLocal() = %v, want %v", cfg.IsLocal(), tt.isLocal) - } - if cfg.IsOCP() != tt.isOCP { - t.Errorf("IsOCP() = %v, want %v", cfg.IsOCP(), tt.isOCP) - } - }) - } -} - diff --git a/internal/k8s/kubectl.go b/internal/k8s/kubectl.go deleted file mode 100644 index db442cb..0000000 --- a/internal/k8s/kubectl.go +++ /dev/null @@ -1,243 +0,0 @@ -package k8s - -import ( - "bytes" - "context" - "encoding/base64" - "fmt" - "io" - "os" - "os/exec" - "strings" - "time" - - "github.com/stackrox/harness-openshell/internal/status" - "gopkg.in/yaml.v3" -) - -var transientErrors = []string{ - "connection refused", - "connection reset", - "timeout", - "etcd leader changed", - "the object has been modified", - "unable to connect to the server", - "TLS handshake timeout", - "i/o timeout", -} - -// Runner abstracts kubectl/helm/oc operations for testing. -type Runner interface { - RunKubectl(ctx context.Context, args ...string) (string, error) - RunKubectlOpts(ctx context.Context, opts KubectlOpts) (string, error) - RunKubectlQuiet(ctx context.Context, args ...string) error - RunHelm(ctx context.Context, args ...string) error - RunOC(ctx context.Context, args ...string) error - ApplyYAML(ctx context.Context, resources ...map[string]any) error - SecretExists(ctx context.Context, name string) bool - GetSecretField(ctx context.Context, secretName, field string) ([]byte, error) - GetJSONPath(ctx context.Context, resource, jsonpath string) (string, error) - NamespaceExists(ctx context.Context, ns string) bool - GetServiceNodePort(ctx context.Context, svcName string, containerPort int) (int, error) - GetNodeInternalIP(ctx context.Context) (string, error) -} - -type Client struct { - kubeconfig string - namespace string -} - -func New(kubeconfig, namespace string) *Client { - return &Client{kubeconfig: kubeconfig, namespace: namespace} -} - -type KubectlOpts struct { - Args []string - Stdin io.Reader - Quiet bool -} - -func (c *Client) RunKubectl(ctx context.Context, args ...string) (string, error) { - return c.RunKubectlOpts(ctx, KubectlOpts{Args: args}) -} - -func (c *Client) RunKubectlOpts(ctx context.Context, opts KubectlOpts) (string, error) { - args := opts.Args - if c.namespace != "" && !containsFlag(args, "-n", "--namespace") { - args = append([]string{"-n", c.namespace}, args...) - } - if c.kubeconfig != "" { - args = append([]string{"--kubeconfig", c.kubeconfig}, args...) - } - - status.Cmd("kubectl", args...) - - var lastErr error - for attempt := range 3 { - cmd := exec.CommandContext(ctx, "kubectl", args...) - if opts.Stdin != nil { - cmd.Stdin = opts.Stdin - } - - var stdout, stderr bytes.Buffer - cmd.Stderr = &stderr - if opts.Quiet { - cmd.Stdout = io.Discard - } else { - cmd.Stdout = &stdout - } - - lastErr = cmd.Run() - if lastErr == nil { - return strings.TrimSpace(stdout.String()), nil - } - - errOutput := stderr.String() + " " + lastErr.Error() - if !isTransient(errOutput) { - return "", fmt.Errorf("kubectl %s: %s", strings.Join(args, " "), strings.TrimSpace(stderr.String())) - } - - if attempt < 2 { - delay := time.Duration(1<&2 -exit 1 -`) - c := New("", "") - _, err := c.RunKubectl(context.Background(), "get", "ns", "nonexistent") - if err == nil { - t.Error("expected error") - } -} - -func TestRunKubectl_RetryOnTransient(t *testing.T) { - dir := t.TempDir() - counterFile := filepath.Join(dir, "count") - os.WriteFile(counterFile, []byte("0"), 0o644) - - writeStub(t, `#!/bin/bash -COUNT=$(cat `+counterFile+`) -COUNT=$((COUNT + 1)) -echo $COUNT > `+counterFile+` -if [ $COUNT -lt 2 ]; then - echo "connection refused" >&2 - exit 1 -fi -echo "ok" -`) - c := New("", "") - out, err := c.RunKubectl(context.Background(), "get", "pods") - if err != nil { - t.Fatalf("RunKubectl: %v (should have retried)", err) - } - if out != "ok" { - t.Errorf("output = %q", out) - } - data, _ := os.ReadFile(counterFile) - if string(data) != "2\n" { - t.Errorf("expected 2 attempts, got %s", data) - } -} - -func TestRunKubectl_NoRetryOnNonTransient(t *testing.T) { - dir := t.TempDir() - counterFile := filepath.Join(dir, "count") - os.WriteFile(counterFile, []byte("0"), 0o644) - - writeStub(t, `#!/bin/bash -COUNT=$(cat `+counterFile+`) -COUNT=$((COUNT + 1)) -echo $COUNT > `+counterFile+` -echo "resource not found" >&2 -exit 1 -`) - c := New("", "") - _, err := c.RunKubectl(context.Background(), "get", "secret", "missing") - if err == nil { - t.Error("expected error") - } - data, _ := os.ReadFile(counterFile) - if string(data) != "1\n" { - t.Errorf("expected 1 attempt (no retry), got %s", data) - } -} - -func TestRunKubectl_NamespaceInjection(t *testing.T) { - dir := t.TempDir() - argsFile := filepath.Join(dir, "args") - writeStub(t, `#!/bin/bash -printf '%s\n' "$*" > `+argsFile+` -`) - c := New("", "openshell") - c.RunKubectl(context.Background(), "get", "pods") - data, _ := os.ReadFile(argsFile) - args := string(data) - if args != "-n openshell get pods\n" { - t.Errorf("args = %q, expected namespace injection", args) - } -} - -func TestRunKubectl_KubeconfigInjection(t *testing.T) { - dir := t.TempDir() - argsFile := filepath.Join(dir, "args") - writeStub(t, `#!/bin/bash -printf '%s\n' "$*" > `+argsFile+` -`) - c := New("/path/to/kubeconfig", "") - c.RunKubectl(context.Background(), "get", "ns") - data, _ := os.ReadFile(argsFile) - args := string(data) - if args != "--kubeconfig /path/to/kubeconfig get ns\n" { - t.Errorf("args = %q, expected kubeconfig injection", args) - } -} - -func TestApplyYAML(t *testing.T) { - dir := t.TempDir() - stdinFile := filepath.Join(dir, "stdin") - writeStub(t, `#!/bin/bash -cat > `+stdinFile+` -`) - c := New("", "test-ns") - err := c.ApplyYAML(context.Background(), map[string]any{ - "apiVersion": "v1", - "kind": "ServiceAccount", - "metadata": map[string]any{"name": "test-sa"}, - }) - if err != nil { - t.Fatalf("ApplyYAML: %v", err) - } - data, _ := os.ReadFile(stdinFile) - content := string(data) - if !strings.Contains(content, "kind: ServiceAccount") { - t.Errorf("YAML missing ServiceAccount: %s", content) - } - if !strings.Contains(content, "name: test-sa") { - t.Errorf("YAML missing name: %s", content) - } -} - -func TestSecretExists(t *testing.T) { - writeStub(t, `#!/bin/bash -[[ "$*" == *"my-secret"* ]] && exit 0 -exit 1 -`) - c := New("", "default") - if !c.SecretExists(context.Background(), "my-secret") { - t.Error("expected secret to exist") - } - if c.SecretExists(context.Background(), "missing") { - t.Error("expected secret to not exist") - } -} - -func TestIsTransient(t *testing.T) { - if !isTransient("dial tcp: connection refused") { - t.Error("connection refused should be transient") - } - if !isTransient("etcd leader changed") { - t.Error("etcd leader changed should be transient") - } - if isTransient("resource not found") { - t.Error("not found should NOT be transient") - } -} - -func TestRunKubectl_RetryExhausted(t *testing.T) { - dir := t.TempDir() - counterFile := filepath.Join(dir, "count") - os.WriteFile(counterFile, []byte("0"), 0o644) - - writeStub(t, `#!/bin/bash -COUNT=$(cat `+counterFile+`) -COUNT=$((COUNT + 1)) -echo $COUNT > `+counterFile+` -echo "connection refused" >&2 -exit 1 -`) - c := New("", "") - _, err := c.RunKubectl(context.Background(), "get", "pods") - if err == nil { - t.Error("expected error after retry exhaustion") - } - data, _ := os.ReadFile(counterFile) - if string(data) != "3\n" { - t.Errorf("expected 3 attempts, got %s", data) - } -} - -func TestRunKubectlQuiet_DiscardsOutput(t *testing.T) { - writeStub(t, `#!/bin/bash -echo "this should be discarded" -echo "error output" >&2 -`) - c := New("", "") - err := c.RunKubectlQuiet(context.Background(), "get", "pods") - if err != nil { - t.Errorf("RunKubectlQuiet: %v", err) - } -} - -func TestGetSecretField_Valid(t *testing.T) { - writeStub(t, `#!/bin/bash -# Return base64-encoded "hello" -echo -n "aGVsbG8=" -`) - c := New("", "default") - data, err := c.GetSecretField(context.Background(), "my-secret", "data-field") - if err != nil { - t.Fatalf("GetSecretField: %v", err) - } - if string(data) != "hello" { - t.Errorf("data = %q, want hello", string(data)) - } -} - -func TestGetSecretField_InvalidBase64(t *testing.T) { - writeStub(t, `#!/bin/bash -echo -n "not-valid-base64!!!" -`) - c := New("", "default") - _, err := c.GetSecretField(context.Background(), "my-secret", "field") - if err == nil { - t.Error("expected error for invalid base64") - } -} - -func TestGetSecretField_Empty(t *testing.T) { - writeStub(t, `#!/bin/bash -echo -n "" -`) - c := New("", "default") - data, err := c.GetSecretField(context.Background(), "my-secret", "field") - if err != nil { - t.Fatalf("GetSecretField: %v", err) - } - if len(data) != 0 { - t.Errorf("expected empty, got %q", data) - } -} - -func TestNamespaceExists(t *testing.T) { - writeStub(t, `#!/bin/bash -[[ "$*" == *"my-ns"* ]] && exit 0 -exit 1 -`) - c := New("", "") - if !c.NamespaceExists(context.Background(), "my-ns") { - t.Error("expected namespace to exist") - } - if c.NamespaceExists(context.Background(), "missing-ns") { - t.Error("expected namespace to not exist") - } -} - -func TestDefaultNamespace(t *testing.T) { - t.Setenv("OPENSHELL_NAMESPACE", "custom-ns") - if ns := DefaultNamespace(); ns != "custom-ns" { - t.Errorf("DefaultNamespace = %q, want custom-ns", ns) - } -} - -func TestDefaultNamespace_Default(t *testing.T) { - t.Setenv("OPENSHELL_NAMESPACE", "") - if ns := DefaultNamespace(); ns != "openshell" { - t.Errorf("DefaultNamespace = %q, want openshell", ns) - } -} - diff --git a/internal/k8s/mock.go b/internal/k8s/mock.go deleted file mode 100644 index cc89ca6..0000000 --- a/internal/k8s/mock.go +++ /dev/null @@ -1,149 +0,0 @@ -package k8s - -import ( - "context" - "fmt" - "strings" -) - -// MockRunner records calls for testing. Returns preconfigured responses. -type MockRunner struct { - Calls []string - Responses map[string]string // command prefix → stdout response - Errors map[string]error // command prefix → error -} - -func NewMockRunner() *MockRunner { - return &MockRunner{ - Responses: make(map[string]string), - Errors: make(map[string]error), - } -} - -func (m *MockRunner) record(args ...string) string { - call := strings.Join(args, " ") - m.Calls = append(m.Calls, call) - return call -} - -func (m *MockRunner) respond(call string) (string, error) { - for prefix, err := range m.Errors { - if strings.HasPrefix(call, prefix) { - return "", err - } - } - for prefix, resp := range m.Responses { - if strings.HasPrefix(call, prefix) { - return resp, nil - } - } - return "", nil -} - -func (m *MockRunner) RunKubectl(_ context.Context, args ...string) (string, error) { - return m.respond(m.record(args...)) -} - -func (m *MockRunner) RunKubectlOpts(_ context.Context, opts KubectlOpts) (string, error) { - return m.respond(m.record(opts.Args...)) -} - -func (m *MockRunner) RunKubectlQuiet(_ context.Context, args ...string) error { - _, err := m.respond(m.record(args...)) - return err -} - -func (m *MockRunner) RunHelm(_ context.Context, args ...string) error { - _, err := m.respond(m.record(append([]string{"helm"}, args...)...)) - return err -} - -func (m *MockRunner) RunOC(_ context.Context, args ...string) error { - _, err := m.respond(m.record(append([]string{"oc"}, args...)...)) - return err -} - -func (m *MockRunner) ApplyYAML(_ context.Context, resources ...map[string]any) error { - for _, r := range resources { - kind, _ := r["kind"].(string) - m.record("apply-yaml", kind) - } - return nil -} - -func (m *MockRunner) SecretExists(_ context.Context, name string) bool { - call := m.record("secret-exists", name) - _, err := m.respond(call) - return err == nil -} - -func (m *MockRunner) GetSecretField(_ context.Context, secretName, field string) ([]byte, error) { - call := m.record("get-secret-field", secretName, field) - resp, err := m.respond(call) - if err != nil { - return nil, err - } - return []byte(resp), nil -} - -func (m *MockRunner) GetJSONPath(_ context.Context, resource, jsonpath string) (string, error) { - return m.respond(m.record("get-jsonpath", resource, jsonpath)) -} - -func (m *MockRunner) NamespaceExists(_ context.Context, ns string) bool { - call := m.record("namespace-exists", ns) - _, err := m.respond(call) - return err == nil -} - -func (m *MockRunner) GetServiceNodePort(_ context.Context, svcName string, containerPort int) (int, error) { - call := m.record(fmt.Sprintf("get-nodeport %s %d", svcName, containerPort)) - resp, err := m.respond(call) - if err != nil { - return 0, err - } - if resp == "" { - return 30080, nil // default test NodePort - } - var port int - fmt.Sscanf(resp, "%d", &port) - return port, nil -} - -func (m *MockRunner) GetNodeInternalIP(_ context.Context) (string, error) { - call := m.record("get-node-ip") - resp, err := m.respond(call) - if err != nil { - return "", err - } - if resp == "" { - return "172.18.0.2", nil // default test node IP - } - return resp, nil -} - -// HasCall checks if any recorded call starts with the given prefix. -func (m *MockRunner) HasCall(prefix string) bool { - for _, c := range m.Calls { - if strings.HasPrefix(c, prefix) { - return true - } - } - return false -} - -// CallCount returns how many calls start with the given prefix. -func (m *MockRunner) CallCount(prefix string) int { - n := 0 - for _, c := range m.Calls { - if strings.HasPrefix(c, prefix) { - n++ - } - } - return n -} - -// String returns a readable dump of all calls. -func (m *MockRunner) String() string { - return fmt.Sprintf("MockRunner{%d calls: %v}", len(m.Calls), m.Calls) -} diff --git a/internal/openshell/types.go b/internal/openshell/types.go index a39a2af..b2cdba7 100644 --- a/internal/openshell/types.go +++ b/internal/openshell/types.go @@ -3,10 +3,9 @@ package openshell // Target identifies what to connect to. // // Gateway is the OPENSHELL REGISTRATION name — the directory under -// ~/.config/openshell/gateways/ managed by the openshell CLI. It is NOT a -// harness gateway profile (e.g. "openshift", "local-container"); those name -// deployment recipes, not registered gateways. Never pass an agent.AgentConfig -// gateway profile here. +// ~/.config/openshell/gateways/ managed by the openshell CLI, as shown by +// `openshell gateway list`. The harness never provisions gateways; it only +// targets one OpenShell already stood up and the user selected. type Target struct { Gateway string // required; openshell registration name Workspace string // "" defaults to "default" (defaulting owned by sdkclient) diff --git a/main.go b/main.go index 9f868a7..fb5f14a 100644 --- a/main.go +++ b/main.go @@ -8,6 +8,7 @@ import ( "github.com/spf13/cobra" "github.com/stackrox/harness-openshell/cmd" + "github.com/stackrox/harness-openshell/internal/gateway" "github.com/stackrox/harness-openshell/internal/openshell/sdkclient" "github.com/stackrox/harness-openshell/internal/status" ) @@ -17,15 +18,6 @@ var version = "dev" //go:embed profiles/agent-basic.yaml var defaultAgentConfig []byte -//go:embed profiles/gateways/local-container.yaml -var localContainerGatewayProfile []byte - -//go:embed profiles/gateways/helm.yaml -var helmNodeportGatewayProfile []byte - -//go:embed profiles/gateways/openshift.yaml -var helmOpenshiftRouteGatewayProfile []byte - func main() { harnessDir := detectHarnessDir() @@ -54,34 +46,19 @@ func main() { cmd.Version = version cmd.DefaultAgentConfig = defaultAgentConfig - cmd.EmbeddedGatewayProfiles = map[string][]byte{ - "local-container": localContainerGatewayProfile, - "helm": helmNodeportGatewayProfile, - "openshift": helmOpenshiftRouteGatewayProfile, - } root.CompletionOptions.HiddenDefaultCmd = true root.AddCommand( cmd.NewApplyCmd(harnessDir, cli, sdkclient.New), cmd.NewGetCmd(sdkclient.New), cmd.NewDescribeCmd(sdkclient.New), - cmd.NewDeleteCmd(harnessDir, cli, sdkclient.New), - cmd.NewDeployCmd(harnessDir, cli), + cmd.NewDeleteCmd(gateway.New(cli), sdkclient.New), cmd.NewDoctorCmd(harnessDir, cli, sdkclient.New), - cmd.NewInitCmd(harnessDir), + cmd.NewInitCmd(), cmd.NewMigrateCmd(), cmd.NewPlanCmd(harnessDir, sdkclient.New), ) - // Deprecated aliases - teardownCmd := cmd.NewTeardownCmd(harnessDir, cli) - teardownCmd.Hidden = true - teardownCmd.Deprecated = "use 'harness delete' instead" - statusCmd := cmd.NewStatusCmd(harnessDir, cli) - statusCmd.Hidden = true - statusCmd.Deprecated = "use 'harness get agents' instead" - root.AddCommand(teardownCmd, statusCmd) - if err := root.Execute(); err != nil { fmt.Fprintf(os.Stderr, "Error: %v\n", err) os.Exit(1) @@ -116,7 +93,6 @@ func detectHarnessDir() string { } if home, err := os.UserHomeDir(); err == nil { d := filepath.Join(home, ".config", "harness-openshell") - os.MkdirAll(filepath.Join(d, "profiles", "gateways"), 0o755) os.MkdirAll(filepath.Join(d, "profiles", "providers"), 0o755) return d } diff --git a/profiles/README.md b/profiles/README.md index 59e097b..fdc03d4 100644 --- a/profiles/README.md +++ b/profiles/README.md @@ -13,7 +13,6 @@ entrypoint: claude # claude, opencode, bash, or any binary on PATH tty: true # enable TTY (default: true) repo: https://github.com/org/repo # cloned outside sandbox, uploaded to /sandbox/ repo_ref: main # branch, tag, or ref to clone (default: HEAD) -gateway: openshift # target gateway (default: local-container) task: @tasks/review.md # task file passed to entrypoint via -p image: ghcr.io/... # override sandbox image policy: path/to/policy.yaml # network policy file @@ -72,12 +71,14 @@ network_policies: - { host: "api.github.com", port: 443 } ``` -Supported kinds: `agent`, `provider`, `gateway`, `payload` (alias: `config`), `policy`. +Supported kinds: `agent`, `provider`, `payload` (alias: `config`), `policy`. + +A `kind: gateway` document is still accepted for backward compatibility but has +no effect: the harness no longer provisions gateways — it targets whichever +gateway OpenShell already stood up and you selected (`openshell gateway +select`). `harness migrate` warns when it finds one so you can register it with +the openshell CLI instead. ## Providers (`providers/`) See [providers/README.md](providers/README.md). - -## Gateways (`gateways/`) - -See [gateways/README.md](gateways/README.md). diff --git a/profiles/gateways/README.md b/profiles/gateways/README.md deleted file mode 100644 index 931884e..0000000 --- a/profiles/gateways/README.md +++ /dev/null @@ -1,76 +0,0 @@ -# profiles/gateways/ - -Gateway profiles define where and how the OpenShell gateway is deployed. - -## Format - -```yaml -gateway: - type: local # local or remote - platform: ocp # k8s or ocp (remote only) - service: route # route, nodeport, or loadbalancer (remote only) - name: openshell-remote-ocp # gateway name for openshell CLI - mode: direct # direct or launcher (remote only) - -chart: - version: "0.0.110" # Helm chart version (keep in lockstep with the openshell CLI) - -helm: - values: # Helm values passed to openshell chart - server: - auth: - allowUnauthenticatedUsers: true - pkiInitJob: - enabled: true - -addons: - manifests: # additional K8s manifests applied after install - - apiVersion: route.openshift.io/v1 - kind: Route - metadata: - name: gateway - spec: - tls: - termination: passthrough - to: - kind: Service - name: openshell - -ocp: # OpenShift-specific config - scc-privileged: [openshell] # ServiceAccounts needing privileged SCC - scc-anyuid: [openshell] # ServiceAccounts needing anyuid SCC - -secrets: - mtls: openshell-client-tls # K8s Secret containing mTLS client certs -``` - -## Targets - -### `local-container.yaml` -- Podman on your machine - -The default. Requires openshell installed at the pinned version (`make openshell`) and the gateway running (`brew services start openshell` on macOS, `systemctl --user start openshell-gateway` on Linux). No Helm, no K8s. - -### `helm.yaml` -- local kind cluster - -Deploys to a kind cluster. Uses NodePort access (no Ingress needed). TLS disabled for local dev simplicity. Requires `kind create cluster`. - -### `openshift.yaml` -- OpenShift cluster - -Deploys to an OpenShift cluster with Route-based access and mTLS. Requires `oc login` and cluster-admin for SCC grants. - -## Selecting a gateway - -```bash -harness apply -f harness.yaml # uses local (default) -harness apply -f harness.yaml --gateway openshift # uses gateways/openshift.yaml -harness apply -f harness.yaml --gateway helm # uses gateways/helm.yaml -``` - -Agent configs can also set a default gateway: - -```yaml -name: agent -gateway: openshift -``` - -The `OPENSHELL_GATEWAY` env var works as a fallback. diff --git a/profiles/gateways/helm.yaml b/profiles/gateways/helm.yaml deleted file mode 100644 index 5b0ede4..0000000 --- a/profiles/gateways/helm.yaml +++ /dev/null @@ -1,35 +0,0 @@ -# kind gateway — deploys openshell to a local kind cluster. -# -# Prerequisites: -# - kind installed and cluster running: -# kind create cluster --name openshell -# - kubectl context set to the kind cluster -# -# Uses NodePort + node InternalIP for access (no Ingress controller needed). -# Direct mode: no launcher Job, providers registered from workstation. -# No mTLS: the gateway runs HTTP for local dev simplicity. - -gateway: - type: remote - platform: k8s - service: nodeport - name: openshell-kind - mode: direct - -chart: - # Keep in lockstep with the openshell CLI version (see openshift.yaml and the - # OPENSHELL_VERSION pin in .github/workflows/integration.yml). The chart - # version determines the supervisor image tag; if it lags the CLI, sandbox - # create fails with "supervisor session not found" during the ssh/tar upload. - version: "0.0.110" - -helm: - values: - service: - type: NodePort - server: - disableTls: true - auth: - allowUnauthenticatedUsers: true - pkiInitJob: - enabled: true diff --git a/profiles/gateways/local-container.yaml b/profiles/gateways/local-container.yaml deleted file mode 100644 index 23f9602..0000000 --- a/profiles/gateways/local-container.yaml +++ /dev/null @@ -1,9 +0,0 @@ -# Local gateway — runs on your machine via podman or docker. -# -# The openshell gateway must be installed (at the pinned version) and running: -# Install: make openshell # pinned to .openshell-version, matches CI -# Start: brew services start openshell (macOS) -# systemctl --user start openshell-gateway (Linux) - -gateway: - type: local diff --git a/profiles/gateways/openshift.yaml b/profiles/gateways/openshift.yaml deleted file mode 100644 index e4f5bea..0000000 --- a/profiles/gateways/openshift.yaml +++ /dev/null @@ -1,56 +0,0 @@ -# OpenShift gateway — deploys openshell to an OCP cluster. -# -# Prerequisites: -# - oc login to the target cluster -# - KUBECONFIG set or default context pointing to OCP - -gateway: - type: remote - platform: ocp - service: route - name: openshell-remote-ocp - -chart: - # Keep in lockstep with .openshell-version / gateway.MinOpenShellVersion. A - # chart behind the CLI pin can leave the inference gRPC Unimplemented and skews - # the supervisor image. Verified live at 0.0.110 on OCP 2026-08-25. - version: "0.0.110" - -helm: - values: - image: - pullPolicy: Always - supervisor: - image: - pullPolicy: Always - securityContext: - runAsUser: null - runAsNonRoot: null - server: - sandboxImagePullPolicy: Always - auth: - allowUnauthenticatedUsers: true - pkiInitJob: - enabled: true - -addons: - manifests: - - apiVersion: route.openshift.io/v1 - kind: Route - metadata: - name: gateway - spec: - tls: - termination: passthrough - to: - kind: Service - name: openshell - port: - targetPort: grpc - -ocp: - scc-privileged: [openshell, openshell-sandbox] - scc-anyuid: [openshell] - -secrets: - mtls: openshell-client-tls diff --git a/test/lib/provision.sh b/test/lib/provision.sh new file mode 100644 index 0000000..51a653a --- /dev/null +++ b/test/lib/provision.sh @@ -0,0 +1,218 @@ +#!/usr/bin/env bash +# Gateway provisioning for the integration tests, using UPSTREAM tooling only. +# +# The harness no longer provisions gateways (PR7b): provisioning is OpenShell's +# job. This library reproduces, in bash, exactly what the retired `harness +# deploy` did — the OpenShell installer already stands up the local gateway, and +# `helm install openshell` + `openshell gateway add/select` stand up cluster +# gateways. Sourced by test-flow.sh and kind-lifecycle.sh. +# +# Requires in the environment: CLI (the openshell binary name), and for cluster +# flows: kubectl, helm, a reachable cluster via KUBECONFIG. + +# Chart/CRD coordinates — kept in lockstep with the values the retired +# cmd/deploy.go + profiles/gateways/*.yaml used. Version follows +# .openshell-version (the single source of truth), overridable via +# OPENSHELL_CHART_VERSION to match cmd/deploy.go's old behavior. +OPENSHELL_CHART_OCI="${OPENSHELL_CHART_OCI:-oci://ghcr.io/nvidia/openshell/helm-chart}" +OPENSHELL_CRD_URL="${OPENSHELL_CRD_URL:-https://github.com/kubernetes-sigs/agent-sandbox/releases/download/v0.5.0/manifest.yaml}" + +_chart_version() { + if [[ -n "${OPENSHELL_CHART_VERSION:-}" ]]; then + echo "$OPENSHELL_CHART_VERSION" + return + fi + local root ver + root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" + ver="$(cat "$root/.openshell-version" 2>/dev/null | tr -d 'v[:space:]')" + echo "${ver:-0.0.110}" +} + +# provision_local: the OpenShell installer already provisioned and started the +# local gateway (see .github/workflows/integration.yml "Install openshell" + +# "Wait for gateway"). Select the 127.0.0.1 registration and confirm it responds. +provision_local() { + local gw + gw="$("$CLI" gateway list 2>/dev/null | strip_ansi | awk '/127\.0\.0\.1/ {gsub(/^\*/, ""); print $1; exit}')" + if [[ -z "$gw" ]]; then + echo " ERROR: no local (127.0.0.1) gateway registered — is OpenShell installed and running?" >&2 + return 1 + fi + "$CLI" gateway select "$gw" || return 1 + local i + for i in $(seq 1 5); do + "$CLI" inference get &>/dev/null && return 0 + sleep 3 + done + echo " ERROR: local gateway $gw not responding" >&2 + return 1 +} + +# provision_kind: faithful bash port of cmd/deploy.go deployFromConfig()'s +# nodeport path (profiles/gateways/helm.yaml). Assumes KUBECONFIG points at a +# reachable kind cluster with helm available. +provision_kind() { + local ver values np ip i + ver="$(_chart_version)" + + kubectl create ns openshell --dry-run=client -o yaml | kubectl apply -f - || return 1 + kubectl label ns openshell \ + pod-security.kubernetes.io/enforce=privileged \ + pod-security.kubernetes.io/warn=privileged --overwrite || return 1 + + kubectl apply -f "$OPENSHELL_CRD_URL" || return 1 + + values="$(mktemp /tmp/os-kind-values-XXXXXX.yaml)" + cat > "$values" <<'EOF' +service: + type: NodePort +server: + disableTls: true + auth: + allowUnauthenticatedUsers: true +pkiInitJob: + enabled: true +EOF + + local helm_args=(upgrade --install openshell "$OPENSHELL_CHART_OCI" + --namespace openshell --version "$ver" --values "$values") + [[ -n "${HARNESS_OS_IMAGE:-}" ]] && helm_args+=(--set "server.sandboxImage=$HARNESS_OS_IMAGE") + helm "${helm_args[@]}" || { rm -f "$values"; return 1; } + rm -f "$values" + + kubectl rollout status statefulset/openshell -n openshell --timeout=300s || return 1 + + np="$(kubectl get svc openshell -n openshell -o jsonpath='{.spec.ports[?(@.port==8080)].nodePort}')" + ip="$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type=="InternalIP")].address}')" + if [[ -z "$np" || -z "$ip" ]]; then + echo " ERROR: could not resolve NodePort ($np) / node IP ($ip)" >&2 + return 1 + fi + + # kind runs disableTls=true → register plaintext HTTP (skips mTLS/browser auth). + "$CLI" gateway remove openshell-kind 2>/dev/null || true + "$CLI" gateway add "http://$ip:$np" --name openshell-kind --local || return 1 + "$CLI" gateway select openshell-kind || return 1 + + for i in $(seq 1 30); do + "$CLI" inference get &>/dev/null && return 0 + sleep 2 + done + echo " ERROR: kind gateway not reachable after 60s" >&2 + return 1 +} + +# provision_ocp: bash port of deployFromConfig()'s OCP route path +# (profiles/gateways/openshift.yaml). Not run in CI (no OCP cluster there); +# validated locally against real OpenShift. Requires oc/kubectl + helm + a +# route-capable cluster. +provision_ocp() { + local ver domain route_host i + ver="$(_chart_version)" + + kubectl create ns openshell --dry-run=client -o yaml | kubectl apply -f - || return 1 + kubectl label ns openshell \ + pod-security.kubernetes.io/enforce=privileged \ + pod-security.kubernetes.io/warn=privileged --overwrite || return 1 + kubectl apply -f "$OPENSHELL_CRD_URL" || return 1 + + # SCCs (openshift.yaml ocp.scc-*). + local sa + for sa in openshell openshell-sandbox; do + oc adm policy add-scc-to-user privileged -z "$sa" -n openshell || return 1 + done + oc adm policy add-scc-to-user anyuid -z openshell -n openshell || return 1 + + # Route (passthrough) — apply before helm so the PKI SAN can match. + domain="$(kubectl get ingresses.config.openshift.io cluster -o jsonpath='{.spec.domain}')" + if [[ -z "$domain" ]]; then + echo " ERROR: could not determine OpenShift apps domain (is this OCP?)" >&2 + return 1 + fi + route_host="gateway-openshell.$domain" + kubectl apply -n openshell -f - < "$values" <<'EOF' +image: + pullPolicy: Always +supervisor: + image: + pullPolicy: Always +securityContext: + runAsUser: null + runAsNonRoot: null +server: + sandboxImagePullPolicy: Always + auth: + allowUnauthenticatedUsers: true +pkiInitJob: + enabled: true +EOF + local helm_args=(upgrade --install openshell "$OPENSHELL_CHART_OCI" + --namespace openshell --version "$ver" --values "$values" + --set "pkiInitJob.serverDnsNames[0]=$route_host") + [[ -n "${HARNESS_OS_IMAGE:-}" ]] && helm_args+=(--set "server.sandboxImage=$HARNESS_OS_IMAGE") + [[ -n "${HARNESS_OS_PULL_SECRET:-}" ]] && helm_args+=(--set "imagePullSecrets[0].name=$HARNESS_OS_PULL_SECRET") + [[ -n "${HARNESS_OS_SANDBOX_PULL_SECRET:-}" ]] && helm_args+=(--set "server.sandboxImagePullSecrets[0].name=$HARNESS_OS_SANDBOX_PULL_SECRET") + helm "${helm_args[@]}" || { rm -f "$values"; return 1; } + rm -f "$values" + + kubectl rollout status statefulset/openshell -n openshell --timeout=300s || return 1 + + # Extract mTLS bundle from the cluster secret and register the route gateway. + local mtls_dir field + mtls_dir="$HOME/.config/openshell/gateways/openshell-remote-ocp/mtls" + mkdir -p "$mtls_dir" + chmod 700 "$mtls_dir" || return 1 + for field in ca.crt tls.crt tls.key; do + kubectl get secret openshell-client-tls -n openshell \ + -o jsonpath="{.data.$field}" | base64 -d > "$mtls_dir/$field" || return 1 + chmod 600 "$mtls_dir/$field" || return 1 + done + + "$CLI" gateway remove openshell-remote-ocp 2>/dev/null || true + "$CLI" gateway add "https://$route_host:443" --name openshell-remote-ocp --local || return 1 + "$CLI" gateway select openshell-remote-ocp || return 1 + + for i in $(seq 1 30); do + "$CLI" inference get &>/dev/null && return 0 + sleep 2 + done + echo " ERROR: OCP gateway not reachable after 60s" >&2 + return 1 +} + +# teardown_cluster: replaces `harness delete --k8s`. helm uninstall + gateway +# deregister + namespace delete. Best-effort (idempotent). +# +# Waits for the namespace to finish deleting before returning: the kind and +# fresh-OCP flows reprovision immediately after teardown, and a create/apply/helm +# against a still-`Terminating` namespace fails. Best-effort — warns rather than +# fails if a finalizer stalls deletion past the timeout. +teardown_cluster() { + local gw_name="${1:-}" + helm uninstall openshell -n openshell 2>/dev/null || true + [[ -n "$gw_name" ]] && "$CLI" gateway remove "$gw_name" 2>/dev/null || true + kubectl delete ns openshell --ignore-not-found --wait=false 2>/dev/null || true + local i + for i in $(seq 1 60); do + kubectl get ns openshell &>/dev/null || return 0 + sleep 2 + done + echo " WARN: namespace 'openshell' still terminating after 120s" >&2 +} diff --git a/test/suite/run.sh b/test/suite/run.sh index 5e37e69..7213be5 100755 --- a/test/suite/run.sh +++ b/test/suite/run.sh @@ -96,38 +96,42 @@ wait_sandbox() { echo "=== Config parsing ===" +# Config parsing is validated offline via `apply -o yaml`, which resolves the +# full config and renders it without touching a gateway. (`--dry-run` is a +# pre-apply preflight that now requires a reachable, selected gateway, so it is +# not an offline config-parse check.) run_test "parse: minimal agent (no providers)" \ - "$HARNESS" apply --dry-run -f "$CONFIGS/agent-minimal.yaml" + "$HARNESS" apply -o yaml -f "$CONFIGS/agent-minimal.yaml" run_test "parse: multi-provider agent" \ - "$HARNESS" apply --dry-run -f "$CONFIGS/agent-multi-provider.yaml" + "$HARNESS" apply -o yaml -f "$CONFIGS/agent-multi-provider.yaml" run_test "parse: task agent" \ - "$HARNESS" apply --dry-run -f "$CONFIGS/agent-task.yaml" + "$HARNESS" apply -o yaml -f "$CONFIGS/agent-task.yaml" run_test "parse: multi-doc harness yaml" \ - "$HARNESS" apply --dry-run -f "$CONFIGS/harness-multidoc.yaml" + "$HARNESS" apply -o yaml -f "$CONFIGS/harness-multidoc.yaml" run_test "parse: harness with policy" \ - "$HARNESS" apply --dry-run -f "$CONFIGS/harness-with-policy.yaml" + "$HARNESS" apply -o yaml -f "$CONFIGS/harness-with-policy.yaml" run_test "parse: default agent (no -f)" \ - "$HARNESS" apply --dry-run + "$HARNESS" apply -o yaml run_test "parse: custom provider profile" \ - "$HARNESS" apply --dry-run -f "$CONFIGS/agent-groq.yaml" + "$HARNESS" apply -o yaml -f "$CONFIGS/agent-groq.yaml" run_test "parse: harness with payloads" \ - "$HARNESS" apply --dry-run -f "$CONFIGS/harness-with-payloads.yaml" + "$HARNESS" apply -o yaml -f "$CONFIGS/harness-with-payloads.yaml" run_test "output: kind: payload in -o yaml" \ bash -c '"$1" apply -o yaml -f "$2" | grep "kind: payload"' _ "$HARNESS" "$CONFIGS/harness-with-payloads.yaml" run_test_fail "parse: nonexistent file rejects" \ - "$HARNESS" apply --dry-run -f "/nonexistent/agent.yaml" + "$HARNESS" apply -o yaml -f "/nonexistent/agent.yaml" run_test_fail "parse: invalid yaml rejects" \ - bash -c 'f=$(mktemp); echo "name: [broken" > "$f"; "$1" apply --dry-run -f "$f"; rc=$?; rm -f "$f"; exit $rc' _ "$HARNESS" + bash -c 'f=$(mktemp); echo "name: [broken" > "$f"; "$1" apply -o yaml -f "$f"; rc=$?; rm -f "$f"; exit $rc' _ "$HARNESS" echo "" @@ -152,18 +156,15 @@ run_test "output: env template preserved (not expanded)" \ echo "" -# ── 3. CLI Flags (4 tests) ────────────────────────────────────── +# ── 3. CLI Flags (3 tests) ────────────────────────────────────── echo "=== CLI flags ===" run_test "flags: --agent default" \ - "$HARNESS" apply --dry-run --agent default + "$HARNESS" apply -o yaml --agent default run_test_fail "flags: --agent nonexistent rejects" \ - "$HARNESS" apply --dry-run --agent nonexistent - -run_test_fail "flags: --gateway + --gateway-profile rejects" \ - "$HARNESS" apply --dry-run --gateway local --gateway-profile /dev/null + "$HARNESS" apply -o yaml --agent nonexistent run_test_fail "flags: delete with no args rejects" \ "$HARNESS" delete diff --git a/test/test-flow.sh b/test/test-flow.sh index 69bedd8..df0d58a 100755 --- a/test/test-flow.sh +++ b/test/test-flow.sh @@ -75,6 +75,9 @@ strip_ansi() { sed 's/\x1b\[[0-9;]*m//g' } +# Upstream gateway provisioning (the harness no longer provisions — PR7b). +source "$SCRIPT_DIR/test/lib/provision.sh" + PASS=0 FAIL=0 TOTAL_START=$(date +%s) @@ -179,15 +182,12 @@ summary() { test_errors() { echo "=== test: error scenarios ===" - step_fail "nonexistent profile" harness apply --gateway local-container --agent nonexistent + step_fail "nonexistent profile" harness apply --agent nonexistent - if $REUSE_GATEWAY; then - step "teardown (first)" harness delete --sandboxes --providers - step "teardown (second)" harness delete --sandboxes --providers - else - step "teardown (first)" harness delete --sandboxes --providers --k8s - step "teardown (second)" harness delete --sandboxes --providers --k8s - fi + # Best-effort sandbox/provider cleanup (skips cleanly if no active gateway). + # Cluster teardown lives in each target's flow via teardown_cluster. + step "teardown (first)" harness delete --sandboxes --providers + step "teardown (second)" harness delete --sandboxes --providers echo "" } @@ -200,12 +200,12 @@ test_local() { echo "=== test-flow: local-container ($mode) ===" step "teardown" harness delete --sandboxes --providers - step "deploy" harness deploy local-container + step "provision" provision_local step "gateway reachable" "$CLI" inference get # up auto-registers providers when missing local sandbox_name="test-agent" - step "sandbox create (up)" harness apply --gateway local-container --name "$sandbox_name" $AGENT_FLAG "$PROFILE" + step "sandbox create (up)" harness apply --name "$sandbox_name" $AGENT_FLAG "$PROFILE" sandbox_verify "$sandbox_name" step "sandbox delete" "$CLI" sandbox delete "$sandbox_name" @@ -218,7 +218,7 @@ test_local() { echo "" echo "=== test: missing providers ===" step "teardown providers" harness delete --providers - step "up with no providers" harness apply --gateway local-container --name test-noprov + step "up with no providers" harness apply --name test-noprov step "cleanup" harness delete --sandboxes fi @@ -259,12 +259,12 @@ test_kind() { return fi - step "teardown" harness delete --sandboxes --providers --k8s - step "deploy" harness deploy helm + step "teardown" teardown_cluster openshell-kind + step "provision" provision_kind step "gateway reachable" "$CLI" inference get local sandbox_name="test-kind" - step "sandbox create" harness apply --gateway helm --name "$sandbox_name" $AGENT_FLAG "$PROFILE" + step "sandbox create" harness apply --name "$sandbox_name" $AGENT_FLAG "$PROFILE" sandbox_verify "$sandbox_name" if ! $NO_PROVIDERS; then @@ -273,7 +273,8 @@ test_kind() { step "sandbox delete" "$CLI" sandbox delete "$sandbox_name" - step "teardown (clean)" harness delete --sandboxes --providers --k8s + step "teardown (sandboxes+providers)" harness delete --sandboxes --providers + step "teardown (cluster)" teardown_cluster openshell-kind echo "" } @@ -290,13 +291,14 @@ test_ocp() { step "teardown sandboxes+providers" harness delete --sandboxes --providers if ! "$CLI" inference get &>/dev/null; then - step "deploy" harness deploy openshift + step "provision" provision_ocp else step "gateway reachable" "$CLI" inference get fi else - step "teardown" harness delete --sandboxes --providers --k8s - step "deploy" harness deploy openshift + step "teardown (sandboxes+providers)" harness delete --sandboxes --providers + step "teardown (cluster)" teardown_cluster openshell-remote-ocp + step "provision" provision_ocp fi local sandbox_name @@ -305,7 +307,7 @@ test_ocp() { step "sandbox create" harness apply -f test/ci-agent.yaml --name "$sandbox_name" else sandbox_name="agent" - step "sandbox create (up)" harness apply --gateway openshift --name "$sandbox_name" + step "sandbox create (up)" harness apply --name "$sandbox_name" fi sandbox_verify "$sandbox_name" @@ -314,7 +316,8 @@ test_ocp() { if $REUSE_GATEWAY; then step "teardown (sandboxes+providers)" harness delete --sandboxes --providers else - step "teardown (clean)" harness delete --sandboxes --providers --k8s + step "teardown (sandboxes+providers)" harness delete --sandboxes --providers + step "teardown (cluster)" teardown_cluster openshell-remote-ocp fi }